Is HPDCACHE_CMO_INVAL_NLINE known to work?

[Intuity]

Hello - I'm attempting to use the invalidation cache management operation to evict a cache line, and I'm having a number of problems. I'm wondering if address specific invalidation is being tested and is known to work?

In my testing I am just caching and invalidating a single address.

• I read an address X - this triggers a corresponding downstream response to fill the line and then the read response comes back normally;
• When sending a request I do not get a response to CMO operations, (need_rsp is tied high);
• When invalidating an address I can see the state machine iterate through CMOH_IDLE, CMOH_WAIT_MSHR_RTAB_EMPTY, CMOH_INVAL_SET, and back to CMOH_IDLE - each state taking just one cycle;
• Then when I read back address X, I do not see another downstream response and the cache immediately responds (i.e. the invalidation has not taken effect).

The documentation seems to be out-of-date around the configuration:

From page 53:

If CONF_HPDCACHE_SUPPORT_CMO is set to 0, the HPDcache does not support CMOs from requesters. See Cache Management Operation (CMO) Handler for more details.

• As far as I can tell from grepping, CONF_HPDCACHE_SUPPORT_CMO is no longer a parameter.

From page 55 table 6.3:

• Signal: core_req_o.op Value: HPDCACHE_REQ_CMO - HPDCACHE_REQ_CMO is not defined anywhere
• Signal: core_req_o.size Value: HPDCACHE_CMO_INVAL_NLINE - HPDCACHE_CMO_INVAL_NLINE is not defined anywhere

Instead it seems that these two may have been unified into HPDCACHE_REQ_CMO_INVAL_NLINE and now the size field is irrelevant?

[cfuguet]

Hello @Intuity,

Thank you for the issue report.

You are right, the documentation is out-of-date. I'm sorry for that. I delivered the release 5.0.0 of the RTL without all the corresponding modifications in the documentation.

Currently the documentation is more aligned with tag 4.0.0. But I'm working on that. Again sorry for that.

To directly answer your question: yes, the invalidation and other CMOs are validated and known to work. However, as you mentioned, the way to encode the operation has changed.

Now the CMO opcode is encoded in the request opcode (as below), and the request size is now irrelevant.

Currently, there is no acknowledge for a CMO (even if the need_rsp = 1). Initially, I did not see the usage of an acknowledge as CMOs are blocking in the HPDcache (the pipeline is stalled while processing CMOs). But now I actually see the usage in some systems, so I will add this in a future version.

    typedef enum logic [4:0] {
        HPDCACHE_REQ_LOAD                  = 5'h00,
        HPDCACHE_REQ_STORE                 = 5'h01,
        // RESERVED                        = 5'h02,
        // RESERVED                        = 5'h03,
        HPDCACHE_REQ_AMO_LR                = 5'h04,
        HPDCACHE_REQ_AMO_SC                = 5'h05,
        HPDCACHE_REQ_AMO_SWAP              = 5'h06,
        HPDCACHE_REQ_AMO_ADD               = 5'h07,
        HPDCACHE_REQ_AMO_AND               = 5'h08,
        HPDCACHE_REQ_AMO_OR                = 5'h09,
        HPDCACHE_REQ_AMO_XOR               = 5'h0a,
        HPDCACHE_REQ_AMO_MAX               = 5'h0b,
        HPDCACHE_REQ_AMO_MAXU              = 5'h0c,
        HPDCACHE_REQ_AMO_MIN               = 5'h0d,
        HPDCACHE_REQ_AMO_MINU              = 5'h0e,
        // RESERVED                        = 5'h0f,
        HPDCACHE_REQ_CMO_FENCE             = 5'h10,
        HPDCACHE_REQ_CMO_PREFETCH          = 5'h11,
        HPDCACHE_REQ_CMO_INVAL_NLINE       = 5'h12,
        HPDCACHE_REQ_CMO_INVAL_ALL         = 5'h13,
        HPDCACHE_REQ_CMO_FLUSH_NLINE       = 5'h14,
        HPDCACHE_REQ_CMO_FLUSH_ALL         = 5'h15,
        HPDCACHE_REQ_CMO_FLUSH_INVAL_NLINE = 5'h16,
        HPDCACHE_REQ_CMO_FLUSH_INVAL_ALL   = 5'h17
    } hpdcache_req_op_t;

[Intuity]

Hello @cfuguet

Thanks for the fast response - however, I'm pretty certain there's a bug in hpdcache_cmo.sv.

I've been poking around trying to understand how the FSM performs the invalidation and I've noticed a discrepancy between INVAL_ALL and INVAL_BY_NLINE where only the latter sets dir_inval_set_o.

I've modified the code as shown below and now the invalidation is working correctly for me - do you agree this is a bug?

[Intuity]

Actually, referring to line 183 I see that dir_inval_set_o = cmoh_set_q - but this is pipelined through a cycle too late for it to match up with the dir_inval_o strobe

[cfuguet]

I think you are right.

The line you added is missing:

dir_inval_set_o = cmoh_set

I introduced this bug when adding the support for flushes....

Would you like to do a pull request ?

[cfuguet]

Fix by PR #46