wallet/db.c, wallet/wallet.c: Add a partial index to speed up startup.

[ZmnSCPxj]

Closes: #4901

Tested by EXPLAIN QUERY PLAN on sqlite3; #4901 shows the result from
@whitslack doing a similar partial index on PostgreSQL on his ~1000 chan
node.

[ZmnSCPxj]

It would be really nice if we could put in the constants here instead of hardcoding the constant values 9 and 19. Any ideas how to do this? Could use a function for this migration so we can db_bind_int, but in principle a function is not necessary and would be too heavyweight...

[vincenzopalazzo]

maybe it is something stupid and I'm missing the point, but we could define a #define RCVD_REMOVE_ACK_REVOCATION 9 somewhere?

Just worry if it is a valid type of db_bind_int

[ZmnSCPxj]

They are already defined here:

lightning/common/htlc_state.h

Line 22 in a9e261f

RCVD_REMOVE_ACK_REVOCATION,

lightning/common/htlc_state.h

Line 36 in a9e261f

SENT_REMOVE_ACK_REVOCATION,

The #define would duplicate the above declaration, and if not carefully ordered relative to the above, would cause a syntax error. We also prefer to use the above enum declaration if possible.

[whitslack]

Rather than asserting the values of these constants in a comment, you could use a compile-time assertion. I see you have a BUILD_ASSERT macro presumably for that purpose. wallet/wallet.h does contain assertions of the numeric values of these enumerators, but you could duplicate those assertions here in wallet/db.c at zero cost to make absolutely certain that the enumerators retain the values you're expecting.

[ZmnSCPxj]

It is not really about asserting the values, it is more that stylistically, I would prefer not to see hardcoded numbers in code. Ideally I would be able to do something like tal_fmt("CREATE INDEX blah blah WHERE hstate NOT IN (%d, %d);;", RCVD_REMOVE_ACK_REVOCATION, SENT_REMOVE_ACK_REVOCATION) but that cannot work as this is a static data structure and this is C, sigh. C++ initializers...

I think BUILD_ASSERT would have to be outside the db_migrations array, too. And the point is not to check the numbers (it is already done elsewhere, as you note) but as a reminder to future readers of the code what these hardcoded 9 and 19 are. If BUILD_ASSERT has to be done outside of db_migrations, then as db_migrations grows, the explanation of the hardcoded 9 and 19 becomes more distant from the query. In that respect a comment is superior as it can sit right next to that query in perpetuam (and the check is already done elsewhere).

[ZmnSCPxj]

Then you should at least put a comment adjacent to the assertions in wallet/wallet.h to help future coders understand what they will be breaking if they violate the assertions.

We already have a warning comment in common/htlc_state.h for this, which is more important since this is where the constants are actually defined:

lightning/common/htlc_state.h

Lines 5 to 8 in a9e261f

	/*
	* /!\ The generated enum values are used in the database, DO NOT
	* reorder or insert new values (appending at the end is ok) /!\
	*/

[ZmnSCPxj]

BUILD_ASSERT_OR_ZERO works:

/* example.c */
#include<stddef.h>

#define BUILD_ASSERT_OR_ZERO(cond) \
        (sizeof(char [1 - 2*!(cond)]) - 1)

enum example {
        EXAMPLE_ZERO,
        EXAMPLE_ONE
};

struct db_entry {
        const char *query;
        const char *foo;
};

struct db_entry db_migrations[] = {
        { "string"
        + BUILD_ASSERT_OR_ZERO(EXAMPLE_ZERO == 0)
        + BUILD_ASSERT_OR_ZERO(EXAMPLE_ONE == 1),
          NULL
        },
};

int main() {
        return 0;
}

$ gcc -c example.c

Still suboptimal though since we do not really want to assert, we want to not have to use hardcoded numbers in the source code...

[ZmnSCPxj]

Except clang hates it...

wallet/db.c:868:2: error: adding 'unsigned long' to a string does not append to the string [-Werror,-Wstring-plus-int]
        + BUILD_ASSERT_OR_ZERO( 9 == RCVD_REMOVE_ACK_REVOCATION)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
wallet/db.c:868:2: note: use array indexing to silence this warning
        + BUILD_ASSERT_OR_ZERO( 9 == RCVD_REMOVE_ACK_REVOCATION)
        ^
        [                                                       ]

bleah.....

[whitslack]

What about &"string"[BUILD_ASSERT_OR_ZERO(…)]?

[ZmnSCPxj]

Current version already does that.

[ZmnSCPxj]

Modified to use BUILD_ASSERT_OR_ZERO Still non-ideal since ideally the numbers 9 and 19 should never appear in the source code.

[cdecker]

ACK ZmnSCPxj@7c63ce4