Return error for excluded secondary-index keys

[CriesofCarrots]

Problem

As of #17110, nodes running with secondary account indexes can explicitly exclude/include keys from the index. If a key is excluded, a full account scan will run. But since these keys are being excluded explicitly because there are too many indexed values for callers to handle, a full account scan is impractical -- it can only perform worse.

Summary of Changes

• Add custom rpc error for excluded keys
• Check keys for exclusion in Rpc before attempting indexed scan -- This operation could be done here:

  solana/runtime/src/accounts_db.rs

  Line 2283 in 597373f

  if !self.account_indexes.include_key(key) {

  where we are already doing the check. However this would require a bunch of churn to change type signatures from Vec to Result, and anyway it seems more reasonable for the rpc service to decide whether to support the full-accounts scan, instead of a hard block in the runtime.

[CriesofCarrots]

@mvines , if you wouldn't mind, does the rpc error seem satisfactory?
Also, it occurred to me that we could make the error more general, and return it for any attempted full accounts scan if some criteria is met: secondary indexes are in place (this seems too heavy-handed to me), some secondary-index keys are included/excluded, or validator arg is supplied.

[jeffwashington]

I think the way you did this is ok. However, the .account_indexes is specified in the config to rpc also from validator. I specifically had to chop out the include/exclude info. So, you could just keep a copy of the include/exclude info in rpc directly. There are 2 sources of truth then, but we already do that with account_indexes.

[jeffwashington]

here is where I exclude the include/exclude info

[CriesofCarrots]

Ah, I failed to track where the verbose/concise account_indexes were split. Yeah, let's do that; saves any runtime helpers needed.

[CriesofCarrots]

@jeffwashington done here: 32118af
Can you take a look?

I considered changing the type to Arc<AccountSecondaryIndexes> to reduce to only one source of truth, but the level of churn was very high. Let me know if you prefer that, though.

[jeffwashington]

perfect.
I prefer not an arc myself. The secondary config is created once at startup and never changed and looked up on each modification to the accounts index. 2 copies is ok with me.

[codecov]

Codecov Report

Merging #17193 (32118af) into master (3e0c0ab) will decrease coverage by 0.0%.
The diff coverage is 14.0%.

@@            Coverage Diff            @@
##           master   #17193     +/-   ##
=========================================
- Coverage    82.7%    82.7%   -0.1%     
=========================================
  Files         421      421             
  Lines      117859   117892     +33     
=========================================
+ Hits        97509    97522     +13     
- Misses      20350    20370     +20     

[jeffwashington]

lgtm