AssertionError: Cannot compute a linearization with respect to 0 arguments

[VineetTambe]

I am trying to run the sf.optimizer:
here is how I create my optimizer:

optimizer = Optimizer(
    factors=factors,
    optimized_keys=_optimized_keys,
    debug_stats=True,  # Return problem stats for every iteration
    params=Optimizer.Params(verbose=True),  # Customize optimizer behavior
)

But the above line throws the following assertion error:
AssertionError: Cannot compute a linearization with respect to 0 arguments

This is how create _optimized_keys,
_optimized_keys = [f"nodes[{i}]" for i in range(_num_nodes)]

All my sub functions do have arguments. So I do not understand where exactly am I going wrong.

Please let me know if you need any more logs or sys info.
Thanks in advance!

[aaron-skydio]

Mostly it would be useful to know the factors you're using, and the stack trace - it would be good to update that error message to be a ValueError instead of an assert, and include 1) the name of the factor, if present, and 2) probably self.inputs.keys() (the names of the inputs to the factor)

[VineetTambe]

My residual function takes in pose and 2 nodes as arguments (I am not sure how to use epsilon here)

This is how I am building my factors:

def build_factors(num_nodes: int) -> T.Iterator[Factor]:
   
    for i in range(num_nodes):
        yield Factor(
            residual=residual,
            keys=[f"poses[{i}]", 
                  f"nodes[edges[{i}][0]]", 
                  f"nodes[edges[{i}][1]]", 
                #   "epsilon"
                  ],
        )
    

And this is how I am adding the keys:

_optimized_keys = [f"nodes[{i}]" for i in range(1, _num_nodes)]

Stacktrace:

AssertionError                            Traceback (most recent call last)
Cell In[124], line 2
      1 # Create the optimizer
----> 2 optimizer = Optimizer(
      3     factors=factors,
      4     optimized_keys=_optimized_keys,
      5     debug_stats=True,  # Return problem stats for every iteration
      6     params=Optimizer.Params(verbose=True),  # Customize optimizer behavior
      7 )

File [~/symForce/env/lib/python3.10/site-packages/symforce/opt/optimizer.py:167](https://file+.vscode-resource.vscode-cdn.net/home/vrex/symForce/symforce_ws/pose_graph_opt/src/~/symForce/env/lib/python3.10/site-packages/symforce/opt/optimizer.py:167), in Optimizer.__init__(self, factors, optimized_keys, params, debug_stats, include_jacobians)
    164     # We compute the linearization in the same order as `optimized_keys`
    165     # so that e.g. columns of the generated jacobians are in the same order
    166     factor_opt_keys = [opt_key for opt_key in optimized_keys if opt_key in factor.keys]
--> 167     numeric_factors.append(factor.to_numeric_factor(factor_opt_keys))
    168 else:
    169     # Add unique keys to optimized keys
    170     self.optimized_keys.extend(
    171         opt_key
    172         for opt_key in factor.optimized_keys
    173         if opt_key not in self.optimized_keys
    174     )

File [~/symForce/env/lib/python3.10/site-packages/symforce/opt/factor.py:266](https://file+.vscode-resource.vscode-cdn.net/home/vrex/symForce/symforce_ws/pose_graph_opt/src/~/symForce/env/lib/python3.10/site-packages/symforce/opt/factor.py:266), in Factor.to_numeric_factor(self, optimized_keys, output_dir, namespace, sparse_linearization)
...
--> 712 assert which_args, "Cannot compute a linearization with respect to 0 arguments"
    714 # Ensure the previous codegen has one output
    715 assert len(list(self.outputs.keys())) == 1

AssertionError: Cannot compute a linearization with respect to 0 arguments

[bradley-solliday-skydio]

Hey Vineet. It looks like the reason you're getting an error is because none of the keys in your factors actually match the keys you constructed your Optimizer with. Ex: "nodes[edges[4, 0]]" != "nodes[4]".

If there is actually an edges variable whose values are indices for "nodes", then perhaps what you mean to do is build your factors like this:

def build_factors(num_nodes: int) -> T.Iterator[Factor]:
   
    for i in range(num_nodes):
        yield Factor(
            residual=residual,
            keys=[f"poses[{i}]", 
                  f"nodes[{edges[i][0]}]",  # <-- this line is different
                  f"nodes[{edges[i][1]}]",  # <-- this line is different
                #   "epsilon"
                  ],
        )

so that, if edges[4][0] == 2, then ["poses[4]", "nodes[2]", f"nodes[{edges[4][1]}]"] would have non-empty intersection with _optimized_keys ["nodes[0]", "nodes[1]", "nodes[2]", "nodes[3]", "nodes[4]", ...]

More generally, in order for, say, the 2nd argument of a residual to be optimized, the second key in the factor you construct from it (so "nodes[edges[4][0]]") would have to exactly match one of the keys in _optimized_keys that you construct the Optimizer with.

Conceptually, if none of the keys match, then the factor wouldn't actually do anything. Incidentally, what happens is an AssertionError is raised, as you saw.

[bradley-solliday-skydio]

Regarding Aaron's comment, I agree that the current error message is inadequate. I'd like to modify Optimizer.__init__ to add this check:

        for factor in factors:
            if isinstance(factor, Factor):
                if optimized_keys is None:
                    raise ValueError(
                        "You must specify keys to optimize when passing symbolic factors."
                    )
                # We compute the linearization in the same order as `optimized_keys`
                # so that e.g. columns of the generated jacobians are in the same order
                factor_opt_keys = [opt_key for opt_key in optimized_keys if opt_key in factor.keys]
                # PROPOSED ADDITION vvv
                if len(factor_opt_keys) == 0:
                    raise ValueError(
                        f"Factor {factor.name} has no arguments (keys: {factor.keys}) in "
                        + f"optimized_keys ({optimized_keys})."
                    )
                # END PROPOSED ADDITION ^^^
                numeric_factors.append(factor.to_numeric_factor(factor_opt_keys))

[VineetTambe]

Okay got it thanks - basically my error was a key mismatch error.

[VineetTambe]

I had a basic question -
Lets say my factors have keys nodes[edges[i][0]], nodes[edges[i][1]]
Will I still be optimising for keys nodes[i] or would my optimisation_keys be nodes[edges[i][0]]?

[bradley-solliday-skydio]

Sorry for the delay (didn't notice your last message). But it's the latter, your optimization keys would need to be nodes[edges[i][0]], as the Optimizer just does a simple string compare of the optimization_keys with the factors keys to figure out which ones to optimize.