API: make CategoricalDtype.__eq__ stricter

[jbrockmendel]

We allow for partially-initialized CategoricalDtypes to compare as equal to fully-initialized ones, leading to some surprising behaviors. The one that I stumbled over was this:

cat = pd.Categorical(["a", "b", "c"], ordered=True)
dtype = pd.CategoricalDtype()

>>> cat.dtype == dtype
True

cat2 = cat.astype(dtype)

>>> cat2.dtype == cat.dtype
False

We should stop special-casing categories=None for this purpose.

[jreback]

this was for back compat

iirc there should be some issue refs in the dtypes def

but agreed we should fix this

[jbrockmendel]

The other surprise I found when going to fix this was that pandas_dtype("categorical") has ordered=None while CategoricalDtype() has ordered=False

[jorisvandenbossche]

I think in general I think this "uninitialized" CategoricalDtype object is to support the general "categories" dtype. So if we want to make the dtype more strict, we will need to come up with an alternative way to support this.
(that's not to say that there might be something buggy in the specific example case you show, but that's for astyping an array that is already categorical)

[jbrockmendel]

Closed by #38516