Questions about Job Declaration Protocol

[plebhash]

A place for Q&A around the SV2 Job Declaration Protocol

[pavlenex]

Great idea, we should have this for roles as well and all other parts as a stack. Since JD is the one attracting the most questions, I will pin this discussions so we can have a single place where people can ask/comment/suggest.

[plebhash]

can IdentifyTransactions happen in completely independently from ProvideMissingTransactions?

or is it assumed that they will always happen in sequence?

[Fi3]

they are 2 different thing. IdentifyTransactions it means that server can not identify the txsid from the short list, so client send the long ids instead if the short one in response. ProvideMissingTransactions means that server do not know the txs data, so client must send the txs data. So they do not happen in sequence ProvideMissingTransactions is very common, we never it an IdentifyTransactions

[plebhash]

During our weekly dev call on April 23rd, @rrybarczyk raised an issue. I'll try to re-write it here to the best of my abilities, and please @rrybarczyk + @Fi3 correct if I misrepresented it somehow.

Basically, right now a miner/JDC will propose a template via DeclareMiningJob. If the pool/JDS rejects this template, they will send back a DeclareMiningJob.Error.

@rrybarczyk argued that if the pool/JDS is rejecting because of txs coming from addresses that it is not willing to mine (e.g.: OFAC), then ideally the protocol should allow the miner/JDC to be notified of this reason.

If miner/JDC simply gets a generic DeclareMiningJob.Error, they would potentially get confused because they don't really know the reason for this template being rejected. They would probably end up switching pools.

If miner/JDC knows that their template was rejected because of a specific tx/address, then they would have the chance to make the decision on whether they want to keep on this pool/JDS or switch over to another one.

[TheBlueMatt]

While I don't really have a strong feeling about error messages, it's probably worth pointing out that the pool can generally just send an email. Some in-protocol error message will probably be either length-limited or displayed in a clunky fashion. If a pool wants a user to run some fork of Bitcoin core with censorship, they presumably also want to give step-by-step instructions on how to do so, which they'll probably email because it'll be a longer doc with links.

Whether there's an error message or not, users will still automatically switch to fallback pool(s), so it doesn't change a lot, IMO, but it may be a small UX improvement.

[Fi3]

This is more an implementation detail, a pool is free to use any 255 bytes string as error code https://github.com/stratum-mining/sv2-spec/blob/main/03-Protocol-Overview.md#35-error-codes.

If the miner send a tx that the pool can not mine, something at the miner level must be off so IMO the pool should close the connection rather the let the miner consume electricity for "invalid" job. (this is ofc an implementation detail as well)

The pool can send an error before closing the connection, or/and an email ecc ecc.

[rrybarczyk]

Since @Fi3 pointed out that there is a flexible 255 byte string in the error message, I agree it does not make sense to make a specific error message for this case at the protocol level. I do think that in the implementation, we should provide a custom error message to address if a miner includes a transaction in their block template that the pool is unable to accept.

Working with prod pools/ASICs, there is nothing more frustrating that receiving errors messages with useless information that does not tell me why an error occurred. Would like to avoid that with SRI as much as possible/where it makes sense.

Also, I actually have no idea the probability of a miner selecting a transaction that a pool cannot accept. Is it once a day? Once a year? The frequency of this occurrence will tell us how much energy we actually need to put into this. I suspect it is not that often, but that is not based on any real data. Also, if it is not that often, then we should highlight this when questions from users or external miners arise. A few people unaffiliated with SRI have raised this as a concern, so if we can turn around and say "well the probability of a miner including an OFAC transaction in their block is 0.01%" (or some negligible percentage), then imo that covers all bases and requires no additional dev work (but we should still include a specific message using the flexible 255 bytes in the implementation regardless I think).

[Fi3]

IMO the propability should be 0% since a miner (when is mining with and OFAC pool) should use an OFAC txs selector. Otherwise there is (almost) no point in doing txs declaration, and the miner could just let the pool do the tx selection.

[TheBlueMatt]

Working with prod pools/ASICs, there is nothing more frustrating that receiving errors messages with useless information that does not tell me why an error occurred

That seems like an issue with the specific pool - if they want their miners to filter, they need to tell their users exactly which filtering patches they want their users to run, and its on them to provide a specific error message pointing to the specific code they're demanding. I'm not really clear on what should be done at the spec level here, rather pools get to put a string there, and they should think about what string they put there and we should make sure SRI surfaces that string to the miner so they can go have a human read it.

[plebhash]

recently while chatting with @TheBlueMatt he asked:

how does the pool currently spot-check the work miners are mining on? Or can it?
Can it still request the transactions being worked on/get the block template before the block is found?
...
If a pool wants to check that miners are actually mining valid blocks, at least probabilistically

I tried to answer to the best of my abilities (this rabbit whole was the original motivation behind #77). But I couldn't figure out an answer I was confident about.

@Fi3 @rrybarczyk what is your current understanding of this? Also, @TheBlueMatt please feel free to elaborate on your question.

[Fi3]

I don't get the question:

1. you are talking about pool role, or pool in general?
2. are you asking about SRI implementation or generic sv2 protocol?

[plebhash]

hmm I'm not sure, perhaps @TheBlueMatt can elaborate?

I recall this conversation happened during the same days/week when the public discussions around Ocean's empty blocks were happening, so my assumption is that Matt wanted to clarify his understanding around the notion of future jobs (which is somewhat relevant in this context)

perhaps there's more context with more messages from our conversation:

This was my attempt to answer Matt's original question (listed above):

the JD Client sends a DeclareMiningJob message to the JD Server (which is usually means pool, but on SRI it's implemented as a separate executable).

JDS can immediately send a DeclareMiningJob.Success if they blindly trust the JDC, but it could also send a IdentifyTransactions, which the client would then reply with a list of tx hashes on the IdentifyTransactions.Success response. The table on the spec is missing this cell, but the field name is tx_data_hashes

if for some reason the JDS is not able to find some tx by its hash on its mempool, it will send a ProvideMissingTransactions, which JDC will reply with a ProvideMissingTransactions.Success containing the full txs requested.

finally, JDS would acknowledge the Job with a DeclareMiningJob.Success

then he said:

Ah okay, thanks. Was confused for a minute if this got removed based on something Rachel said but glad it wasn’t (cause it’s important!)

Iiuc the miner will start mining on the new job without waiting for DeclareMiningJob.Success, right?

and I answered:

hmm I think that is not the case

from what I understand so far, now on SV2 we have the notion of Future Jobs

so the JDC can simply negotiate a Future Job with JDS, and keep it ready in-cache.

as soon as a SetNewPrevHash arrivesto JDC, the Future Job is sent downstream with this new prevhash
the main point here is that DeclareMiningJob.Success should have arrived a long time before SetNewPrevHash

then Matt said:

this doesn't let the pool easily validate templates, though, they have to validate them without the current block's context

finally, I answered:

hmm so the edge case would be a tx that landed on the current block being included again on the future template, right?

to be honest, maybe Fi3 or Rachel are probably better suited to answer this.
at this point, I'm still wrapping my brain around JD Protocol, unfortunately

and then I came here and wrote this question...

the main open question (at least for me) is what happens in this edge case, where a future job could potentially include a tx that ends up being included in the current block, resulting in wasted PoW on the future job

Apologies for not making it clear enough on my original posting, hopefully this makes it more clear!

[Fi3]

JDS can immediately send a DeclareMiningJob.Success if they blindly trust the JDC

JDS check if have the proposed txs (using the short ids). If there are collision it use IdentifyTransactions. If do not have tx data in mempool it use ProvideMissingTransactions

Iiuc the miner will start mining on the new job without waiting for DeclareMiningJob.Success, right?

Yes as soon as the TP provide a NewTemplate (not future) the JDC will create a job for downstream and send it and in parallel will initiate the job declaration process. When a SetNewPrevHash that refer an already received (and declared) future template we can just send the job downstream.

this doesn't let the pool easily validate templates, though, they have to validate them without the current block's context

Partially true (only for future template, and the pool can still validate something) but this is just an implementation detail every pool can do whatever it want. A pool can say that a future job is ok in the sense that it do not contains txs that the pool do not want to include. A pool could ask downstream to not declare future jobs but I can't see the reason here: (1) if the pool want to censor txs it can do it also for future templates; (2) if the pool is worried about a client mine invalid jobs and still get rewarded, this do not increase the attack surface, as it would be a lot easier and more impactful to just do block withholding; (3) not very concerning IMO but a pool could be worried by bugged clients, the answer is very easy: double check the jobs at a later time.

[plebhash]

On the current SRI implementation, there's no connection between JDS and Pool.

JDS negotiates some job with a JDC, which then starts sending shares to the Pool.

How does the Pool know that the shares under this specific job are valid?

[Fi3]

SetCustomMiningJob

[plebhash]

ok that makes sense:

• JDC sends AllocateMiningJobToken to JDS
• JDS responds with a mining_job_token via AllocateMiningJobToken.Success
• JDC uses mining_job_token to send DeclareMiningJob to JDS
• JDS responds with new_mining_job_token via DeclareMiningJob.Success
• JDC uses new_mining_job_token when sending SetCustomMiningJob to Pool

but it still leaves the question partially open

there is no communication between JDS and Pool, so how does Pool know which mining_job_token/new_mining_job_token is valid and which isn't?

is that done via some cryptographic mechanism?

[Fi3]

yep the only reasons for having a B0_255 as token instead of something shorter is for allowing JDC to put cryptographic proof in there. Currently in SRI we sign the tx_hash_list_hash field. Some implementation can just decide to not use it, for example when the pool and jds are the same software or have a communication channel (not sv2 based ofc) .

[plebhash]

the spec has a AllocateMiningJobToken.Success, but no AllocateMiningJobToken.Error

is that intentional, or did we forget to specify the error message?

what happens if JDS is not able to allocate a token for some reason? should JDC just wait until some timeout, and then fallback to another JDS or to solo?

cc @Fi3

[Fi3]

yep if jds do not send tokens we should switch upstream

[plebhash]

on the definition of DeclareMiningJob, we have a field called tx_hash_list_hash.

Its defitinion says:

Hash of the full sequence of SHA256(transaction_data) contained in the transaction_hash_list

• what is transaction_hash_list?
• what is transaction_data?

there's no mention to these variable names anywhere else in the spec

[GitGab19]

I never liked those names, but afaik they are:

• transaction_hash_list --> concatenation of normal tx ids
• transaction_data --> full tx data

So the tx_hash_list_hash should be the SHA256 of the tx ids list. Please @Fi3 correct me if I'm wrong

[plebhash]

I'm still slightly confused, so I'll elaborate my current understanding to hopefully get confirmation.

We're leveraging SipHash for short txids. I assume that is a way to reduce bandwidth consumption, since including full txids would make the message body excessively big.

Is this assumption correct?

---

DeclareMiningJob has the follwing fields:

• tx_short_hash_nonce
• tx_short_hash_list
• tx_hash_list_hash

tx_short_hash_nonce

Aside from the preimage (which is the full txid), SipHash also needs keys to avoid collisions, and that is the main motivation for tx_short_hash_nonce, so that we can take a SHA256 from this nonce and use it as k0 and k1.

tx_short_hash_list

tx_short_hash_list carries the short txids, where each short txid is SipHash(txid, k0, k1).

tx_hash_list_hash

tx_hash_list_hash seems to work as some kind of checksum over tx_hash_list_hash.

I can imagine the following optimistic scenario: JDS already has all txs declared on tx_short_hash_list in its internal mempool, and therefore it's able to calculate whatever checksum is necessary (this step is actually not fully clear to me, so I started another discussion thread dedicated to it).

So JDS uses tx_short_hash_list to construct a list of full txids, in the same order as they were declared on tx_hash_list_hash.

Then, JDS concatenates this list (which is currently being called transaction_hash_list) and takes a SHA256 of this concatenation, which is called tx_hash_list_hash.

Is that correct?

[GitGab19]

Is this assumption correct?

Yes it is.
And I confirm you the flow you described is corrected as well 👍

[plebhash]

Another question about DeclareMiningJob (potentially correlated to IdentifyTransaction and ProvideMissingTransactions).

JDC sends a list of short_txid = SipHash-2-4(txid, k0, k1) to JDS, where:

• txid is the transaction data hash of each tx
• k0 and k1 are defined by SHA256(tx_short_hash_nonce)

JDS can easily calculate k0 and k1 from tx_short_hash_nonce.

But for each short_txid, JDS has no prior information about the corresponding txid. It has a bunch of txid on its own mempool, but it has no way to figure out which one will match the specific short_txid.

So how does JDS solve this? I assume that trial-and-error doesn't really scale well, as the complexity of such algorithm would probably explode.

[plebhash]

if we want the JDS to set unique keys for each client, why not drop the strategy of hashing a nonce in favor of JDS directly sending unique (k0, k1) for each client?

I wonder if (k0, k1) couldn't be included as a new field in AllocateMiningJobToken.Success?

and then just drop tx_short_hash_nonce from DeclareMiningJob

that would avoid the need for adding a new message or extension

I'll wait for @TheBlueMatt input but in case we get consensus, we should consider shipping these changes via #104

[Fi3]

@plebhash you can use ephemeral keys from the noise handshake

[Fi3]

if you keep the mempool (not like what the JDS do know in SRI) having a single nonce for every dowstream is still an advantage btw. If you trow the mempool away and get it again as we do know in JDS having a different nonce for each downstream just do not scale.

[plebhash]

I'm confused.

what is the point of having a nonce if we're not going to hash it?

and if JDS is sending the nonce to JDC, then what are the ephemeral noise keys replacing?

[TheBlueMatt]

JDS having a different nonce for each downstream just do not scale.

Yea, we should run numbers. In general we don't expect JDCs to provide new work all that often (though we do expect a bit of a flood after new blocks), so I'm not entirely convinced its a major issue.

However, indeed, we could allow the server to pick the nonces to avoid this, they just need to rotate the nonces every half second or so (faster than someone can put attacking txn in the mempool)....at which point maybe its not worth it cause you're recalculating a lot anyway.

yep but the pool can specify different nonce for different group of down-streams and easily remove who is attacking the pool

Hmm, yea, we could do fixed per-client nonces, though it introduces a rather nontrivial amount of memory (and background CPU usage) per client, but maybe that's the right tradeoff...

[plebhash]

is it possible to use the same mining_job_token over multiple SetCustomMiningJob (to be sent over different Extended Channels)?

or do we need each SetCustomMiningJob (on each Extended Channel) to have a unique mining_job_token?

[TheBlueMatt]

That's true no matter what, though? Obviously the SetCustomMiningJob message has to match the DeclareMiningJob message in all cases, but I don't see how that impacts the job_id field?

[Fi3]

agree job_id do not need to be the same, they can be the same since we send jobs on different channels, but maybe better have different ids. So I would say SHOULD be different?

[TheBlueMatt]

IMO it's up to the machines involved. The pool can set the job_id on the mining level and the proxy on the downstream level. If they want it the same they can do that, if they don't care they don't care.

[Fi3]

What I'm saying is that in the above case: where we send the same SetCustomMiningJob trough different channels with the pool. When the pool answer with SetCustomMiningJob.Success can set whatever job_id wants, same id is fine since we are on different channels, about the SHOULD be different maybe not, I'm sure that they can be different. Agree that the proxy can do whatever he want on downstream ids.

[plebhash]

Why does the job need to be the same? The job can be different or not based on whether the pool/client consider it a different job.

I see your point.

On Mining Protocol cleanup PR I propose introducing the idea of "layers" (although the context is Extended Extranonce, which is only tangentially related to this point), where the search space is gradually narrowed down as work trickles downstream towards workers. I feel this point is somewhat related to that, since each layer can decide to manage work-job mapping under different strategies.

However, I still feel that for most simplified scenarios, it is desirable to avoid fragmenting job_id distribution as much as possible.