Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Official Agent Suggestions Thread #353

Open
11 of 48 tasks
Tracked by #55
0xdevalias opened this issue Jun 1, 2014 · 356 comments
Open
11 of 48 tasks
Tracked by #55

Official Agent Suggestions Thread #353

0xdevalias opened this issue Jun 1, 2014 · 356 comments

Comments

@0xdevalias
Copy link
Member

0xdevalias commented Jun 1, 2014

(Started from discussions in #344)

Official 'suggest an agent' thread for Huginn!!

Have a cool idea for an agent that you want to see implemented?

  • Firstly, please check to see if the agent is already on the list.
  • Also have a quick look through the Issues to make sure it's not already being worked on.
  • Comment on this thread and we will add it to the list below!
  • If the agent is already on this list but you still want to throw extra support behind it, feel free to comment with your +1

Want to work on one of the agents on the list?

  • Create a new issue/pull request to discuss it and reference this issue so we can reference it on the list below and mark it as being in progress.
  • If there is already an issue reference and it's closed, tag one of us to reopen the issue for you.

Agent Suggestions

(Note: This list should be maintained in alphabetical order, ticked items mean in progress/actively being worked on, ticked items should be moved to 'in progress' list)

In Progress

Backlog

Done

Similar services to Huginn (for inspiration)

@0xdevalias
Copy link
Member Author

@cantino / @dsander I can see this checklist getting pretty long/unruly pretty quickly, and perhaps management a bit painful. Do you think an official Huginn Agent Suggestion Trello board might be a good idea? (would have a similar setup to above, can still reference GitHub issues, and should be a bit cleaner/easier to manage)

@0xdevalias 0xdevalias mentioned this issue Jun 1, 2014
@CloCkWeRX
Copy link
Contributor

Various agents I'd be interested in contributing to (have existing ruby doing parts of this, which might only need UI/adapting)

  • Google tasks agent (So when I add an email as a google task, I can send it to trello)
  • Trello agent
  • Xero agent (So when I book leave, it can be added to a calendar)

Much bigger integrations/possibilities:

  • Ninjablocks integration - http://ninjablocks.com/pages/api-docs - a mix of Node, Physical sensors and MQTT (newer versions) lets you fire events for door opening, xbmc playing or actuate power switches. Has rudimentary rules engine built in; but rather than re-invent the wheel something like huginn could go a long way. Useful because it speaks 433mhz.
  • TheThingSystem integration - http://thethingsystem.com/ - NodeJS + Raspberry Pi + a load of plugins. Integrates with everything from lightbulbs, aircons, to Telsa Model S. Much more developer focused at the moment.

@cantino
Copy link
Member

cantino commented Jun 2, 2014

These all sound very interesting @CloCkWeRX, we'd love to have your involvement!

@0xdevalias
Copy link
Member Author

@CloCkWeRX Definitely a fan of Trello (#346 related), and Xero could be interesting too (does that mean you're an aussie too (Canberra based myself), or do they do international stuff?)

@CloCkWeRX
Copy link
Contributor

Am an Aussie (SA based). I have a few days free, so I'll spend a bit getting my local dev env happy. I'll probably focus on the xero part first - what I've got works with the payroll API for who's taking leave; though there's plenty of other bits and pieces that are interesting if you a small/medium business (invoice raised, payment made events?).

@0xdevalias
Copy link
Member Author

Cool :)

I've done a little with Xero through work. There are 3 auth methods for the API: private (public/private key), public (Oauth, key only valid for 30mins) and partner (Oauth, key doesn't expire)

The most useful one for something like Huginn is obviously partner, but there are a bunch of hoops and stuff to jump through to get that one as I understand it.

Will definitely be interested to see what you come up with!

@steph-sysinfo
Copy link

Hi all, I would be interested in Skedpal agent with similar functionnality as provided in Zapier.

@Sieboldianus
Copy link

Sieboldianus commented Feb 7, 2023

Not a user of Huginn, so far, but I came here because it looks like a possible route for my automation problem:

I have a number of monthly manual tasks that I could not automate so far.

One of them is getting PDF from login-protected websites, saving pdfs in specific folders with naming conventions (renaming etc.) and uploading those to my nextcloud.

There are many different websites - the most painful are banking accounts, of which I have 5, with overall 12 PDFs to download & rename each month. I think this is quite difficult to automate, since banking sites have 2FA. There are websites like Spectre or saltedge (suggested by Firefly III) that allow you to provide your full credentials and they will scrape your bank account - but I would give up all banking security to be able to use this.

So I am aiming to start with a simpler one: My phone company PDFs. They are behind a simple login form (user & pw) and uploaded for the past 6 months. I always forget to check regularly enough to download all. The url is phonecompanyxyz.com/get_pdf.php?value=-6. Are there any self-hosted apps available that I could use to automate this?

My current idea is to perhaps use bash & cron in a custom script via curl:

  1. Loading the initial login page to get cookies & CSRF token
  2. Submitting email address with cookies (POST)
  3. Pressing "Log In" after submitting password with cookies (POST)
  4. Downloading PDF, renaming
  5. Uploading PDF to Nextcloud via API/Curl

This sounds a painful lot of work - maybe Huginn could be an alternative? I read that saving PDF files or attaching binary results to events is not possible #433 so far. So maybe I could use a custom Huginn Webhook to login, and get the file via a bash script?

@gingerbeardman
Copy link
Contributor

gingerbeardman commented Feb 7, 2023

@Sieboldianus personally, I would go the bash+cron route.

It's likely your bank website uses JavaScript, so you couldn't use the plain Website Agent anyway, you'd have to use a Post Agent to browserless/phantomjs etc. feeding the results into a Website Agent, which complicates things. You'd have to get the PDF link and auth credentials, and then pass to a Shell Command Agent (which is locked by default in Huginn config) to do the download.

I just don't see how Huginn is making your life easier in any way here. Unless this sounds easier to you?!

@Sieboldianus
Copy link

Thanks so much @gingerbeardman for the answer. This was my conclusion, too, this morning. And I went the Selenium route. And it works (shared my code here).

You are right, I am not sure how much could be added with Huginn. Some pieces still come to my mind:

  • Have a "Selenium" Agent, where all the generic parameters are configured (including connection to remote Docker-Chromedriver) and only the python-selenium-webparsing code needs to be added, depending on the website. In my example, you can see this is not a lot of code: 1. Login 2. Download PDF
  • Have some kind of PDF-Upload Agent or Backup Agent that would upload results (PDFs) to (e.g.) Nextcloud, using Curl

Yet, I expect there is not much need for such a feature update. I'll stick with cron+bash+python+selenium+curl for the moment.

@BoBeR182
Copy link

BoBeR182 commented Jun 3, 2023

ChatGPT / Other LLM agent support

@zombiehoffa
Copy link

It would be awesome to have a nextdns api agent (https://nextdns.github.io/api/) especially to stream logs and sound alarms on specific events.

@adamus1red
Copy link

Being able to pull data from Twitch would be handy, i.e. notifications of streams going live, etc

@hihouhou
Copy link

Being able to pull data from Twitch would be handy, i.e. notifications of streams going live, etc

https://github.com/hihouhou/huginn_twitch_agent ?

@mascimax
Copy link

@gingerbeardman
Copy link
Contributor

gingerbeardman commented Feb 15, 2024

@mascimax seems that is standard RESTful so already possible, eg. using the Post Agent

@ReVoLt112
Copy link

@hihouhou your influxdb agent, merged into official release would be nice!

https://github.com/hihouhou/huginn_influxdb_write_data_agent

@gingerbeardman
Copy link
Contributor

@ReVoLt112 can't it be used as it is? why does it need to be merged into official?

@hihouhou
Copy link

hihouhou commented Feb 19, 2024

@ReVoLt112 can't it be used as it is? why does it need to be merged into official?

https://rubygems.org/gems/huginn_influxdb_write_data_agent :-) you can add it like other agents/gems needed in your .env

@virtadpt
Copy link
Collaborator

@ReVoLt112 can't it be used as it is? why does it need to be merged into official?

It would be really nice to see some new official agent types added? Having lots of third-party ones is fine, that's a healthy ecosystem, but there's something to be said for seeing some new stuff in the main project as well.

@gingerbeardman
Copy link
Contributor

Personally I'd prefer to see bugs fixed and things not break with each update.

@ReVoLt112
Copy link

@ReVoLt112 can't it be used as it is? why does it need to be merged into official?

It would be really nice to see some new official agent types added? Having lots of third-party ones is fine, that's a healthy ecosystem, but there's something to be said for seeing some new stuff in the main project as well.

Is there a tutorial for implementing 3rd party agents into docker-compose?

@hihouhou
Copy link

@ReVoLt112 can't it be used as it is? why does it need to be merged into official?

It would be really nice to see some new official agent types added? Having lots of third-party ones is fine, that's a healthy ecosystem, but there's something to be said for seeing some new stuff in the main project as well.

Is there a tutorial for implementing 3rd party agents into docker-compose?

do like me, you only have to generate a new huginn image with updated .env file -> https://github.com/hihouhou/docker-huginn/blob/master/.env#L181 your remove the comment and add huginn_influxdb_write_data_agent

@gangrif
Copy link

gangrif commented Aug 26, 2024

Im new to Huginn, so forgive me if this is either already solved, or for some reason impossible.

But i'd like to use Huginn for some social media monitoring. This is easy for open networks like mastodon where everythings an RSS feed, but not so easy for Facebook, Instagram, and Threads. Is there any chance that agents to read and maybe post to, these networks might be possible?

@gingerbeardman
Copy link
Contributor

Yes, it's possible. Reading RSS is built-in. As is GETing/POSTing to a API end point.

If there are no API end-points then auth will be hard work, but still possible.

If your only option is to interact with a web page, then Huginn doesn't run JavaScript so you need to use it alongside things like Browserless, etc.

@gangrif
Copy link

gangrif commented Aug 26, 2024

Yes, it's possible. Reading RSS is built-in. As is GETing/POSTing to a API end point.

If there are no API end-points then auth will be hard work, but still possible.

If your only option is to interact with a web page, then Huginn doesn't run JavaScript so you need to use it alongside things like Browserless, etc.

Thanks for the response.
I know that the services I am asking about have API's, Or at least I believe they do. I don't really know if ive got the know-how to make my own agent though, at that level. I think my first question is, has anyone done this already? My searching has come up with a lot of nothing.

Specifically I'm looking for the ability to use the Meta API to read facebook, insta, and threads. bonus if I can write back to them as well to post things on my own accounts.

@gingerbeardman
Copy link
Contributor

gingerbeardman commented Aug 26, 2024

I've never made a single Agent in all my time with Huginn, everything I do is as a Scenario using the built-in Agents. You can think of a Scenario as a workflow of Agents.

People used to do it when Twitter was more open, as there was a built-in Twitter Agent.

If you can't find what you want in issues on this repo (sadly it's not well maintained by the owners, so it's a mess) then the next best place to ask are the Element group (see readme) or https://www.reddit.com/r/selfhosted/comments/fmky18/huginn_agent_mageathread/

I doubt you'll find exactly what you want, particularly for multiple services.

@gangrif
Copy link

gangrif commented Aug 26, 2024

I've never made a single Agent in all my time with Huginn, everything I do is as a Scenario using the built-in agents.

People used to do it when Twitter was more open, as there was a built-in Twitter Agent.

If you can't find what you want in issues on this repo (sadly it's not well maintained by the owners, so it's a mess) then the next best place to ask are the Element group (see readme) or https://www.reddit.com/r/selfhosted/comments/fmky18/huginn_agent_mageathread/

I doubt you'll find exactly what you want, particularly for multiple services.

Thanks again! Ill have a look.

@0xdevalias
Copy link
Member Author

I know that the services I am asking about have API's, Or at least I believe they do.

Facebook locked down it's useful public APIs for reading personal feeds/etc many years ago (around the time of the first election manipulation stuff), so you would basically be having to reverse engineer the private API's that the website itself is using. And since Elon bought Twitter, they have similarly locked down their useful APIs, or made them prohibitively expensive for most average people, so again you'd probably be resorting to reverse engineering the private ones their site uses.

I have some scattered notes across some of my GitHub gists, but it wouldn't really be the most straightforward task unfortunately:

@gangrif
Copy link

gangrif commented Aug 27, 2024

I know that the services I am asking about have API's, Or at least I believe they do.

Facebook locked down it's useful public APIs for reading personal feeds/etc many years ago (around the time of the first election manipulation stuff), so you would basically be having to reverse engineer the private API's that the website itself is using. And since Elon bought Twitter, they have similarly locked down their useful APIs, or made them prohibitively expensive for most average people, so again you'd probably be resorting to reverse engineering the private ones their site uses.

I have some scattered notes across some of my GitHub gists, but it wouldn't really be the most straightforward task unfortunately:

* https://gist.github.com/0xdevalias/0e723b6c46fec7947a3119de9dd5045d#random-facebook-react--relay-javascript-snippets

* https://gist.github.com/0xdevalias/8885b10795eb3267b703ed5943087953#notes-on-apiuserscript-to-improve-twitter-notifications-timeline

How do social management platforms like buffer, hootsuite, zoho, do it then? At least for posting.

@917huB
Copy link

917huB commented Aug 27, 2024

subscribe to an enterprise grade API?

@virtadpt
Copy link
Collaborator

@gangrif Facebook's API is a bad joke; really the only thing you can do (after fighting with it for some time) is post to a group that you own (which is probably not what you want to do). You'd have to buy access to their enterprise integration API if you wanted to do anything more interesting.

I'm pretty sure that Instagram is in the same boat. rss-bridge can generate feeds for particular usernames, hashtags, or locations on Instagram so unless you want to post to it, it would be worth checking that out.

Threads claims to have an API (https://developers.facebook.com/docs/threads/) but I've not messed with it so I don't know if it's worth fighting with or not.

@virtadpt
Copy link
Collaborator

@gingerbeardman Check out the Phantom Js Cloud Agent.

@virtadpt
Copy link
Collaborator

@gangrif They pay for access to the Enterprise API.

@black-roland
Copy link

@gangrif That's probably not what you're looking for but check out Google Alerts. It allows you to create an RSS feed of new content indexed by Google and that new content can be filtered by keywords or domain names.

@gingerbeardman
Copy link
Contributor

@gingerbeardman Check out the Phantom Js Cloud Agent.

It's on my list, but everything just works with browserless I don't have a reason to change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests