Add ManyToManyField will break migrate #150
Comments
|
What's the version? |
|
I'm seeing the same issue and i'm using version aerich==0.5.3. Output is this: |
|
I join, faced with the same problem, version 0.5.3 |
|
@aquinary I'm getting the same issue and |
Yes. Need to delete all the tables that are affected by the migration (including those that are linked), then init-db is worker normally. |
|
I'm also about to switch to sqlalchemy
…On Tue, Jun 8, 2021 at 8:19 PM Aquinary ***@***.***> wrote:
@aquinary <https://github.com/Aquinary> I'm getting the same issue and
init-db isn't resolving it for me instead of resolving the error it's
giving me another error as AttributeError: 'NoneType' object has no
attribute 'pop' on doing aerich migrate
Yes. Need to delete all the tables that are affected by the migration
(including those that are linked), then init-db is worker normally.
This is acceptable in dev, but it can't be a solution in production, so I
switched to sqlalchemy.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#150 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANXV22XVWPB3N2P6IKK6TMTTRYUXZANCNFSM43Q7NJKA>
.
|
|
Are you feeling comfortable with SQLAlchemy now?
…On Tue, Jun 8, 2021 at 8:33 PM FIRDOUS BHAT ***@***.***> wrote:
I'm also about to switch to sqlalchemy
On Tue, Jun 8, 2021 at 8:19 PM Aquinary ***@***.***> wrote:
> @aquinary <https://github.com/Aquinary> I'm getting the same issue and
> init-db isn't resolving it for me instead of resolving the error it's
> giving me another error as AttributeError: 'NoneType' object has no
> attribute 'pop' on doing aerich migrate
>
> Yes. Need to delete all the tables that are affected by the migration
> (including those that are linked), then init-db is worker normally.
> This is acceptable in dev, but it can't be a solution in production, so I
> switched to sqlalchemy.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#150 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ANXV22XVWPB3N2P6IKK6TMTTRYUXZANCNFSM43Q7NJKA>
> .
>
|
It is a little more complex than TORM in the initial setup and is slightly different from TORM in terms of use, but if you look into docs, you can use it quite well. It has its own nuances, which you have to get used to. Something like "more flexibility, but less convenience" + sqlalchemy it's a very good technology to have on resume In SQLAlchemy, you have to write like this and define the necessary related fields in advance There is no syntax like user_from__username in SQLAlchemy. Instead, you should filter the request yourself: But if you get used to it, then the work goes like clockwork and there are no more problems. |
|
Okay, Thank you for explaining it to me.
…On Tue, Jun 8, 2021 at 9:24 PM Aquinary ***@***.***> wrote:
Are you feeling comfortable with SQLAlchemy now?
… <#m_1531964176004411680_>
On Tue, Jun 8, 2021 at 8:33 PM FIRDOUS BHAT *@*.*> wrote: I'm also about
to switch to sqlalchemy On Tue, Jun 8, 2021 at 8:19 PM Aquinary @.*>
wrote: > @aquinary <https://github.com/Aquinary>
https://github.com/Aquinary I'm getting the same issue and > init-db
isn't resolving it for me instead of resolving the error it's > giving me
another error as AttributeError: 'NoneType' object has no > attribute 'pop'
on doing aerich migrate > > Yes. Need to delete all the tables that are
affected by the migration > (including those that are linked), then init-db
is worker normally. > This is acceptable in dev, but it can't be a solution
in production, so I > switched to sqlalchemy. > > — > You are receiving
this because you commented. > Reply to this email directly, view it on
GitHub > <#150 (comment)
<#150 (comment)>>,
> or unsubscribe >
https://github.com/notifications/unsubscribe-auth/ANXV22XVWPB3N2P6IKK6TMTTRYUXZANCNFSM43Q7NJKA
> . >
It is a little more complex than TORM in the initial setup and is slightly
different from TORM in terms of use, but if you look into docs, you can use
it quite well. It has its own nuances, which you have to get used to.
Something like "more flexibility, but less convenience" + sqlalchemy it's a
very good technology to have on resume
Regarding flexibility, for example, in TORM, this code will automatically
create the FK fields user_from_id, user_to_id.
And you can access the associated fields by user_from_ _ username.
class Friend(models.Model):
id = fields.IntField(pk=True)
friend_from = fields.ForeignKeyField('models.User',
related_name='friend_from', description='Отправитель заявки')
friend_to = fields.ForeignKeyField('models.User',
related_name='friend_to', description='Получатель заявки')
status = fields.BooleanField(default=False, description='Статус
подтверждения')
In SQLAlchemy, you have to write like this and define the necessary related fields in advance
class Friend(Base):
*tablename* = 'friends'
user_from_id = Column(Integer, ForeignKey('users.id'), primary_key=True)
user_from = relationship('User', foreign_keys=[user_from_id])
user_to_id = Column(Integer, ForeignKey('users.id'), primary_key=True)
user_to = relationship('User', foreign_keys=[user_to_id])
status = Column(Boolean, default=False)
There is no syntax like user_from__username in SQLAlchemy. Instead, you should filter the request yourself:
request = FriendRequest(user_from__username=username,
user_to__username=request_username) # TORM
request =
FriendRequest(user_from=db.query(User).filter_by(username=username).one(),
user_to=db.query(User).filter_by(username=request_username).one()) #
SQLAlchemy
But if you get used to it, then the work goes like clockwork and there are no more problems.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#150 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANXV22R43WFQZ65JZEX6J23TRY4MFANCNFSM43Q7NJKA>
.
|
|
I'm having the same issue with |
|
I took a bit of a closer look at this and it seems that there's a whole type of action missing in the migration methods. When a key is I didn't have time to test more in depth, but it seemed to me the issue isn't with the differ but with the serialization of the models since the differ isn't able to discriminate the difference between a change in some m2m field and an addition of a new m2m field. But anyway, bumping this thread, it is a big breaking issue. |
|
It's still not fixed, ffs... |
|
Join to issue. aerich==0.5.8 |
|
Ok. I debugged it. Error in usage of method from dictdiffer When it comparing lists of dicts, it's ignore that first list can contains dict from second but at another order. My opinion it's partially mistake of dictdiffer. It should find levenshtein distance for sequences. But it is not solution to fix bug. EXAMPLE:
So, we try to find difference between different m2m's |
Just came across this now.... Had to hack my way around it, by manually changing The following lines in Line 235: This then allows you to actually migrate M2M fields... We shouldn't really have to hack this together, though, for it to work. Another small issue I've found, is if you're using the @long2ice - just checking you've seen this issue? |
|
Still not fixed. aerich==0.6.2. If someone looks for more production ready workaround here it is: # overload in Tortoise model, note: aerich may blow up at model which has M2M field declared
# OR at destination model(!!)
@classmethod
def describe(cls, serializable: bool = True) -> dict:
result = super().describe(serializable)
m2m_order = ('location', 'slaves', 'devicegroups') # << here put your M2M fields names
assert set(m2m_order) == set(cls._meta.m2m_fields)
result['m2m_fields'] = [
cls._meta.fields_map[name].describe(serializable)
for name in m2m_order
]
return resultWorkaround explanation As @dstlny said, the issue is in m2m fields compare. Depending on many things the order of M2M fields may change between migrations (but not have to, I hit this issue after adding third M2M relation to the same model). So to make it working without hacking aerich code or migrations data in aerich database table, I tried to make sure that order of M2M fields will be always the same and newer fields always appears after existing ones. After some time of debugging I found how to achieve this. Aerich is taking model state using Generic fix proposal At the aerich level I think there should be some code which orders |
|
@dstlny thanks, it worked for me |
|
What worked for me was just commenting out the many to many relation. Then first migrate to create the new table and remove the old many_to_many. And then add back the many_to_many for the new table (indeed alembic for sqlalcehemy is a bit more robust here). Still I'm not ditching TORM just yet for this single flaw ;) |
|
I'm having this issue without even making a change to a M2M field on the model, just adding a new CharField. I was able to resolve quickly by patching aerich/migrate.py:257 locally with It seems like the case where action = 'change' isn't covered, and in my case just making sure the dict get is failing quietly, the rest of the migration logic still works. |
|
0.7.1 has bug too. Using dictdiffer is a problem. |
|
Is there any solution? |
1 similar comment
|
Is there any solution? |
|
Change 244 line in migrate.py table = change[0][1].get("through")to if isinstance(change[0][1], str):
for new_m2m_field in new_m2m_fields:
if new_m2m_field['name'] == change[0][1]:
table = new_m2m_field.get('through')
break
else:
table = change[0][1].get("through") |
Hi, could you make a PR? |
Look like permission denied(( |
Why? just fork and make pull request |
Done |
try to fix "Add ManyToManyField will break migrate #150" issues
|
I got the same problem when added 'backward_key' and 'forward_key' to existing ManyToManyField. Fl0kse's solution has not been released yet |
|
@long2ice Is this bug fixed, or fixable ? 3.5+ years passed I believe this is a major flaw, makes tortoise orm and aerich unusable for any projects requires m2m relations Any non-trivial projects probably have some m2m relations |
|
No it's not fixed at all and i just into the problem... just giving it for posterity: for action, option, change in diff(old_m2m_fields, new_m2m_fields):
if isinstance(change[0], bool) or change[0][0] == "db_constraint":
continue
elif isinstance(change[0][1], str):
for new_m2m_field in new_m2m_fields:
if new_m2m_field["name"] == change[0][1]:
table = new_m2m_field.get("through")
break
else:
table = change[0][1].get("through") |
|
Fixed by #390 |
and others
Related code as below:
After I added new ManyToManyField (sys_role_menu, uncomment those lines)，and run
aerich migrate，it throws an AttributeError Exception:The text was updated successfully, but these errors were encountered: