Research on Data Lake philosophy & simplified interfaces as one way to make easier less specialized coworkers to bootstrap intelligent agents

[fititnt]

Hardware is much cheaper than human creative labor. Is not harder to find who is able to do code for a software, but is very hard to find skilled people who could understand, in a context of intelligent agents, like chatbots, WHAT could be useful to be implemented.

I will look if I can find simple implementations to do what is called "data lakes" (https://en.wikipedia.org/wiki/Data_lake) without need set up more complex software that require day to day experts like hadoop & other data warehouse tools on a private project that could pay for the hardware and at some point is very receptive to usage of intelligent agents as support on internal organization.

At the worst case, this would be just a glorified backup of binary data with advanced search options, and the real cost would the research and implementation and not the hardware, but I'm willing to pay for this since if have a small change to be used on other groups.

---

For anyone reading this on the future: even if you read something that appears to be dumb or not optimized (for example, store binary files on a database) read about what this working group is (short description: ChatOps for non-DevOps people Working Group 2018/01). I'm focusing on meta-automations that could be reproduced even from less specialized or skilled people and that could allow people with no data science background be able to use or even design intelligent agents

[fititnt]

A practical ideal example case is a person speaking by voice, or by text: "This is a video file I received from person@example.com." one interface could just push that file with some metadata (email as one strong index, data of upload, who received the file, etc) for a storage. And later allow access with another orders, even if using simple NLP.

Philosophically speaking: we have a Humans + Intelligent Interface + machines working as team, where who do DevOps optimize for the full team. This approach is similar to NoOps, but we try abstract even more to a point where is need coworkers with less required knowledge on the underline technology but higher focus on the human side. (e.g. instead of develop considering support a full team of 10+ software developers in todays NoOps, do the infrastructure for a team of 1 or 2 voluntaries with IT background supporting pro-bono people from different skills, like social scientists, psychologists, social work assistant, etc).

This maybe sounds strange, but I'm pretty sure at some point in the future we will have open source weak artificial inteligences (here a text in Portuguese with something near this). But exist ways to boostrap these more philosophical approaches a lot of years before.

[fititnt]

Created https://github.com/fititnt/chatops-wg-datalake/issues repository.

It's a starting point. Maybe will need to rename it on the future if it become less about data lakes and more about the simplified interfaces for coworkers. For now is a place to write it down what is not done on https://github.com/fititnt/chatops-wg-infra or https://github.com/EticaAI/eticaai-infrastructure, but I'm open to receive new coworkers here at fititnt/chatops-wg who would have users cases who would need and could exchange experiences or I and some coleages could help for the public good.

[fititnt]

Renamed to chatops-wg-data.

New link: https://github.com/fititnt/chatops-wg-data