Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't work, did TGTG change something? #17

Closed
DavyRoswinkel opened this issue Nov 9, 2021 · 64 comments
Closed

Doesn't work, did TGTG change something? #17

DavyRoswinkel opened this issue Nov 9, 2021 · 64 comments

Comments

@DavyRoswinkel
Copy link

Add-on version: 1.8.0
You are running the latest version of this add-on.
System: Home Assistant OS 6.6 (aarch64 / raspberrypi4-64)
Home Assistant Core: 2021.11.2
Home Assistant Supervisor: 2021.10.8

Log:
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing...
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[17:08:48] INFO: Copy setings file
[17:08:48] INFO: Strating Bridge
2021-11-09 17:08:56 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Connecting mqtt
2021-11-09 17:08:56 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Starting loop
2021-11-09 17:08:56 1e14b3fb-tgtg-ha-mqtt-bridge main[308] DEBUG Loop run started
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "toogoodtogo_ha_mqtt_bridge/main.py", line 158, in loop
if not check():
File "toogoodtogo_ha_mqtt_bridge/main.py", line 33, in check
shops = tgtg_client.get_items(page_size=400)
File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 143, in get_items
self._login()
File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 122, in _login
raise TgtgLoginError(response.status_code, response.content)
tgtg.exceptions.TgtgLoginError: (403, b'<title>apptoogoodtogo.com</title><style>#cmsg{animation: A 1.5s;}@Keyframes A{0%{opacity:0;}99%{opacity:0;}100%{opacity:1;}}</style>

Please enable JS and disable any ad blocker

<script>var dd={'cid':'AHrlqAAAAAMAVhKWux7Z960AU_cwAw==','hsh':'1D42C2CA6131C526E09F294FE96F94','t':'fe','r':'b','s':35694,'host':'geo.captcha-delivery.com'}</script><script src="https://ct.captcha-delivery.com/c.js"></script>\n')

Didn't they change the way you login?
In their app you don't need a password, you will get a login link.
Maybe that is what caused this issue, but I could be wrong.

@MaxWinterstein
Copy link
Owner

Yeah, seems like to only allow the email flow with this "two factor" style :(

Already started a discussion at the package I use: ahivert/tgtg-python#119

@DavyRoswinkel
Copy link
Author

Yeah, seems like to only allow the email flow with this "two factor" style :(

Already started a discussion at the package I use: ahivert/tgtg-python#119

A difficult puzzle...😲would love to see a solution..but fear...the fun is over. Like this addon very much!

@MaxWinterstein
Copy link
Owner

Yeah, seems like to only allow the email flow with this "two factor" style :(
Already started a discussion at the package I use: ahivert/tgtg-python#119

A difficult puzzle...😲would love to see a solution..but fear...the fun is over. Like this addon very much!

Hopefully, someone is able to reverse engineer the authentification flow. Sadly not my main field of knowledge. But as this hurts all projects that used the api, regardless the language used, I hope there is someone out there with enough power and will to solve this 😉

@poudenes
Copy link

Chouffy/home_assistant_tgtg#1

other GitHub user with same issue. there was a node-js module posted with a fix. Maybe you can look to it?

@MaxWinterstein
Copy link
Owner

Chouffy/home_assistant_tgtg#1

other GitHub user with same issue. there was a node-js module posted with a fix. Maybe you can look to it?

as far as i see this only fixes the fetch of items, not the login process.

@poudenes
Copy link

@MaxWinterstein
Copy link
Owner

This was closed by the mentioned fix of the node-js module, afaik no fix for login, only for fetching items by adding some header.

@poudenes
Copy link

:(

@poudenes
Copy link

Then my last option:

https://github.com/custom-components/alexa_media_player/wiki

Those guys use 2FA also to connect to Amazon so your Alexa will work in HA.
Maybe you can check their code, or contact them how they did it?

@liudger
Copy link

liudger commented Nov 15, 2021

@MaxWinterstein
Copy link
Owner

MaxWinterstein commented Nov 15, 2021

https://github.com/ahivert/tgtg-python/releases/tag/0.7.0 has a fix

Yes, yay 🥳

Currently there is an PR by @Dielee in progress (#19), seems like this will be solved soon.

@MaxWinterstein
Copy link
Owner

I updated the bridge, and also the addon.

As i am still blocked at TooGoodToGo, pls have a test :)

@jablonski-pawel
Copy link

Hi! Many thanks for @MaxWinterstein for an fast update! I've updated to the 1.9.0, but still I have this same problem with captcha 😭 What am I doing wrong? I had reinstall add-on, and nothing changed 😢 On phone TGTG app it is working fine after check Captcha.

@DavyRoswinkel
Copy link
Author

I updated the bridge, and also the addon.

As i am still blocked at TooGoodToGo, pls have a test :)

Thanks! The first run, showed unknown for the stock of all shops. Te second run...it shows the correct numbers of stock.
It looks like it works again. Will watch it closely this morning.

Thanks again for solving this issue🥳🍾🙋‍♂️

@poudenes
Copy link

poudenes commented Nov 16, 2021

Here updated as well. Log looks Good.

Within first start I see the portions and pickup time coming back. Didn't start twice and no block.

Thanks for the fast and great fix!!

@Dielee
Copy link
Collaborator

Dielee commented Nov 16, 2021

I updated the bridge, and also the addon.

As i am still blocked at TooGoodToGo, pls have a test :)

Maybe it is possible to fetch the captcha file ?
I would have a look into this, but I don't have this type of error...

@poudenes
Copy link

I updated only the Addon and start it. Update bridge? That's inside the the Addon right?

Here its working

@Dielee
Copy link
Collaborator

Dielee commented Nov 16, 2021

Yes, for most of us, also me, its working in 1.8.0. But some people are getting an captcha request.. From app or in the email, I don't now at this time.

@MaxWinterstein
Copy link
Owner

Maybe worth mentioning, as I guess I am not the only one:

My dev machine, and also my iPhone, is ipv6 compatible and uses it to access TGTG and their services.

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.


@Dielee I think solving the captcha and passing the response cookie around should be the solution for this, but I am not sure how to archive this.

@unixfox
Copy link

unixfox commented Nov 16, 2021

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

@MaxWinterstein
Copy link
Owner

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

oh, at least that explains why I wasn't able to get the nodejs version to run also

@DavidDeSloovere
Copy link

Version 1.9.0 of the add-on works for me! Thanks. Already have dinner for tonight.

@unixfox
Copy link

unixfox commented Nov 16, 2021

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

oh, at least that explains why I wasn't able to get the nodejs version to run also

If you are interested in making nodejs working with ipv6 then take a look at this github issue: nodejs/node#6307

@Dielee
Copy link
Collaborator

Dielee commented Nov 16, 2021

@MaxWinterstein is there also an link inside the mail ? Maybe the using an redirect url, like the most oauth. Then, there is an way to archive this.

@MaxWinterstein
Copy link
Owner

The Addon, or any docker container without explicit config I guess, uses ipv4 to communitcate.

NodeJS use ipv4 by default even if IPv6 is available.

oh, at least that explains why I wasn't able to get the nodejs version to run also

If you are interested in making nodejs working with ipv6 then take a look at this github issue: nodejs/node#6307

oh no way, neither do I speak nodejs, nor do I have any interest in ipv6 😄


@MaxWinterstein is there also an link inside the mail ? Maybe the using an redirect url, like the most oauth. Then, there is an way to archive this.

I am not sure if we talk about the same thing? The Captcha is not linked to any mail?

I played around with a flow already, see (see marklagendijk/node-toogoodtogo-watcher#112 (comment))

@Dielee
Copy link
Collaborator

Dielee commented Nov 17, 2021

Maybe let's talk about this in a chat ? Gitter or Discord ?

@DavyRoswinkel
Copy link
Author

DavyRoswinkel commented Nov 17, 2021

And it stopped working for me again.
Had to reboot HA. And then it stopped working with:
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing...
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[13:36:20] INFO: Copy setings file
[13:36:20] INFO: Strating Bridge
2021-11-17 13:36:21 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Connecting mqtt
2021-11-17 13:36:21 1e14b3fb-tgtg-ha-mqtt-bridge main[308] INFO Starting loop
2021-11-17 13:36:21 1e14b3fb-tgtg-ha-mqtt-bridge main[308] DEBUG Logging in with credentials
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "toogoodtogo_ha_mqtt_bridge/main.py", line 202, in loop
tgtg_client.login()
File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 126, in login
raise TgtgLoginError(response.status_code, response.content)
tgtg.exceptions.TgtgLoginError: (404, b'')

I think the end is close...sad🥺

@MaxWinterstein
Copy link
Owner

tgtg.exceptions.TgtgLoginError: (404, b'')

encouraged the 404 here, too. Very strange.

@Dielee
Copy link
Collaborator

Dielee commented Nov 19, 2021

Maybe it is possible to use an Proxy to connect to tgtg servers ?
I would like to open an new branch for this.

@MaxWinterstein
Copy link
Owner

Maybe it is possible to use an Proxy to connect to tgtg servers ? I would like to open an new branch for this.

Had some play with https://github.com/jundymek/free-proxy - that went so so... Most of those IPs are already 'blocked'.

As the 404 issue now seems to persist I might have a try with the email verification flow (already build a tgtg-python here: https://github.com/MaxWinterstein/tgtg-python ), will see if I find some time

@DavidDeSloovere
Copy link

Just to confirm @DarKOrange75 - my 1.9.0 integration is also dead again

@liudger
Copy link

liudger commented Nov 25, 2021

https://github.com/ahivert/tgtg-python/releases/tag/0.8.0

@liudger
Copy link

liudger commented Dec 2, 2021

@MaxWinterstein
Copy link
Owner

I merged the latest PR from @Dielee, now this plugin uses tgtg==0.10.0.

It even works again for me with ipv4 and ipv6!

@DavyRoswinkel can you have a try? (docker image 1.9.1 should be available in a few minutes)

@DavyRoswinkel
Copy link
Author

In the addon store I keep seeing version 1.9.0.

Can you tell me how to update...to docker image 1.9.1?

@MaxWinterstein
Copy link
Owner

Ah, did not see you are using the add-on, my fault.

Need to adjust it a little bit so the now saved tokens will persist across restarts.

Sadly @Dielee did not provide a function to declare the storage position... 😙🎶

@MaxWinterstein
Copy link
Owner

I quickly thew out version 1.9.2 - now both add-on and bridge itself use the same versioning.

Please have a try ✌️

@DavyRoswinkel
Copy link
Author

It logs me in....
It connects to TGTG makes the entities...
But it doesn't give the state/attribute info of the entities.
State is unknown

The second run of the loop
it gives the full info.

Looks like it works again!!!!!
Very happy!

Thanks alot!!

@liudger
Copy link

liudger commented Dec 3, 2021

Could it be that if you failed the first login that it doesn't update the tokens if you try again? (New mail code is send)

@poudenes
Copy link

poudenes commented Dec 3, 2021

Skip my comment. I didnt read correct. I received emails :)

@jablonski-pawel
Copy link

I started the addon received the email from TGTG I clicked the link in the email

and it worked

I had this same. It have to click this button on email, when on LOGs you see "DEBUG Logging in with credentials". Time is a little short - about 1 minute. After that everything works!! It don't need to do this again after reboot HA machine. @MaxWinterstein Thank you!! Great Job!

@poudenes
Copy link

poudenes commented Dec 3, 2021

Installed a clear addon. Did action on email. And then i get a error:

s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-banner.sh: executing... 
-----------------------------------------------------------
 Add-on: TooGoodToGo Home Assistant MQTT Bridge
 Publish TooGoodToGo stock as MQTT messages
-----------------------------------------------------------
 Add-on version: 1.9.2
 You are running the latest version of this add-on.
 System: Home Assistant OS 6.6  (aarch64 / raspberrypi4-64)
 Home Assistant Core: 2021.11.5
 Home Assistant Supervisor: 2021.10.8
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[13:31:35] INFO: Copy setings file
[13:31:35] INFO: Strating Bridge
2021-12-03 13:31:37 f1c878cb-tgtg-ha-mqtt-bridge __main__[309] INFO Connecting mqtt
2021-12-03 13:31:37 f1c878cb-tgtg-ha-mqtt-bridge __main__[309] INFO Starting loop
2021-12-03 13:31:37 f1c878cb-tgtg-ha-mqtt-bridge __main__[309] DEBUG Logging in with credentials
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "toogoodtogo_ha_mqtt_bridge/main.py", line 200, in loop
    tgtg_client.login()
  File "/usr/local/lib/python3.8/site-packages/tgtg/__init__.py", line 139, in login
    self.start_polling(first_login_response["polling_id"])
  File "/usr/local/lib/python3.8/site-packages/tgtg/__init__.py", line 179, in start_polling
    raise TgtgPollingError(
tgtg.exceptions.TgtgPollingError: Max retries (120 seconds) reached. Try again

@MaxWinterstein
Copy link
Owner

Think you must be fast clicking that email, like @jablonski-pawel said. Saw that message too at one point.
Did you click via brower/mail client or at the smartphone were TGTG is also installed?

I might drop some big note and that part.

@poudenes
Copy link

poudenes commented Dec 3, 2021

Ok. I got the error because i open the email on my mobile. Then the TgTg app is opening.
Then the login is not working.

Now i open it on my computer and now its logged in and sensors are back!

@MaxWinterstein
Copy link
Owner

Ok. I got the error because i open the email on my mobile. Then the TgTg app is opening.
Then the login is not working.

Good to know, should mention that! Will try to get some colored message there...

@liudger
Copy link

liudger commented Dec 3, 2021

Ok. I got the error because i open the email on my mobile. Then the TgTg app is opening. Then the login is not working.

Now i open it on my computer and now its logged in and sensors are back!

Yep this indeed works!

@DarKOrange75
Copy link

This issue is closed but it is still not working for me :-(

Add-on: TooGoodToGo Home Assistant MQTT Bridge
Publish TooGoodToGo stock as MQTT messages

Add-on version: 1.9.2
You are running the latest version of this add-on.
System: Home Assistant OS 6.6 (aarch64 / raspberrypi4-64)
Home Assistant Core: 2021.11.5
Home Assistant Supervisor: 2021.10.8

Please, share the above information when looking for help
or support in, e.g., GitHub, forums or the Discord chat.

[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing...
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[20:19:28] INFO: Copy setings file
[20:19:28] INFO: Strating Bridge
2021-12-03 20:19:30 f1c878cb-tgtg-ha-mqtt-bridge main[308] INFO Connecting mqtt
2021-12-03 20:19:30 f1c878cb-tgtg-ha-mqtt-bridge main[308] INFO Starting loop
2021-12-03 20:19:30 f1c878cb-tgtg-ha-mqtt-bridge main[308] DEBUG Logging in with credentials
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "toogoodtogo_ha_mqtt_bridge/main.py", line 200, in loop
tgtg_client.login()
File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 146, in login
raise TgtgLoginError(response.status_code, response.content)
tgtg.exceptions.TgtgLoginError: (403, b'<title>apptoogoodtogo.com</title><style>#cmsg{animation: A 1.5s;}@Keyframes A{0%{opacity:0;}99%{opacity:0;}100%{opacity:1;}}</style>

Please enable JS and disable any ad blocker

<script>var dd={'cid':'AHrlqAAAAAMAgCRU6taq_eIATsZmpw==','hsh':'1D42C2CA6131C526E09F294FE96F94','t':'fe','r':'b','s':35693,'host':'geo.captcha-delivery.com'}</script><script src="https://ct.captcha-delivery.com/c.js"></script>\n')

@DavyRoswinkel
Copy link
Author

Just a thought....

Are you using a password in the configuration?

@DarKOrange75
Copy link

DarKOrange75 commented Dec 3, 2021

nope

Just a thought....

Are you using a password in the configuration?

No I reinstalled the add-on and the configuration does not have a password, just my email address

@DavyRoswinkel
Copy link
Author

nope

Just a thought....

Are you using a password in the configuration?

No I reinstalled the add-on and the Conf file does not have a password, just my email address

That's correct. For me it works now. Don't know the solution for your problem.
Just wait...I am sure Max will help you.

@DarKOrange75
Copy link

nope

Just a thought....
Are you using a password in the configuration?

No I reinstalled the add-on and the Conf file does not have a password, just my email address

That's correct. For me it works now. Don't know the solution for your problem. Just wait...I am sure Max will help you.

Can you please reopen the issue as it is not fixed for everybody? Or should I open a new one?

@DavyRoswinkel
Copy link
Author

I think a new issue works better, as it's a new version of the repo.
Also a little different error log.

@MaxWinterstein
Copy link
Owner

@DarKOrange75 have you been running the old version before? Or a you a new user?

At first lookthis seems like the common issue that your IP has been 'blocked' temporarily.

Are you able to obtain a new IP address, e.g. by rebooting your Router?


@DavyRoswinkel btw, Thanks for the pizza <3

@Vjpiet
Copy link

Vjpiet commented Dec 3, 2021

@DarKOrange75 I experienced the same even with the latest version.
Think my IP was still blocked, using a VPN solved the issue!

@MaxWinterstein Thanks for the great work!

@DarKOrange75
Copy link

@MaxWinterstein and @Vjpiet you are right, I have been running the previous version with the same user, I thought it was disable but was still running so my address/IP might have been blocked. Plus for some reasons my email server was not receiving/rejecting connection emails from tgtg.
I left the addon stopped for the night.
This morning I stopped my mail server and restarted the addon. I received the email on the backup mail server clicked the link and now everything is working again. Thank you all for your help !

@jablonski-pawel
Copy link

This issue is closed but it is still not working for me :-(

Add-on: TooGoodToGo Home Assistant MQTT Bridge

Publish TooGoodToGo stock as MQTT messages

Add-on version: 1.9.2

You are running the latest version of this add-on.
System: Home Assistant OS 6.6 (aarch64 / raspberrypi4-64)
Home Assistant Core: 2021.11.5
Home Assistant Supervisor: 2021.10.8

Please, share the above information when looking for help

or support in, e.g., GitHub, forums or the Discord chat.
[cont-init.d] 00-banner.sh: exited 0. [cont-init.d] 01-log-level.sh: executing... [cont-init.d] 01-log-level.sh: exited 0. [cont-init.d] done. [services.d] starting services [services.d] done. [20:19:28] INFO: Copy setings file [20:19:28] INFO: Strating Bridge 2021-12-03 20:19:30 f1c878cb-tgtg-ha-mqtt-bridge main[308] INFO Connecting mqtt 2021-12-03 20:19:30 f1c878cb-tgtg-ha-mqtt-bridge main[308] INFO Starting loop 2021-12-03 20:19:30 f1c878cb-tgtg-ha-mqtt-bridge main[308] DEBUG Logging in with credentials Exception in thread Thread-3: Traceback (most recent call last): File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/local/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "toogoodtogo_ha_mqtt_bridge/main.py", line 200, in loop tgtg_client.login() File "/usr/local/lib/python3.8/site-packages/tgtg/init.py", line 146, in login raise TgtgLoginError(response.status_code, response.content) tgtg.exceptions.TgtgLoginError: (403, b'<title>apptoogoodtogo.com</title><style>#cmsg{animation: A 1.5s;}@Keyframes A{0%{opacity:0;}99%{opacity:0;}100%{opacity:1;}}</style>

Please enable JS and disable any ad blocker

<script>var dd={'cid':'AHrlqAAAAAMAgCRU6taq_eIATsZmpw==','hsh':'1D42C2CA6131C526E09F294FE96F94','t':'fe','r':'b','s':35693,'host':'geo.captcha-delivery.com'}</script><script src="https://ct.captcha-delivery.com/c.js"></script>\n')

After few hours when it worked, I have this same problem - Captcha is back :/ I reinstalled app, and nothing changed 😢

@treysis
Copy link

treysis commented Jan 6, 2022

If you are interested in making nodejs working with ipv6 then take a look at this github issue: nodejs/node#6307

That's been resolved since NodeJS v17.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests