Plan for continuing with this project

[MichaIng]

@cclauss @Mictronics @jmichault and everyone else, let's make a plan how to continue with this project, now moved to the new @motioneye-project GitHub organisation.

First of all, everyone who has contributed to motionEye before, or wants to do so, please feel free to join in, also I think we can add everyone who is interested to the organisation.

I think we agree that Python 3 support is the first thing to achieve, given that Python 2 has reached EOL since over 2 years already, @jmichault and @Mictronics have Python 3 compatible forks from where we can commit back, and on this repo we have the python3 branch and a bunch of open pull requests for further related fixes.

I personally would go this way:

• Merge the python3 branch: [WIP] Python3 support #1572
• Rebase and merge Python 3 related pull requests
• Rebase and merge https://github.com/Mictronics/motioneye
  • Ah, and that one is here as PR already: Yet another Python3 wip #2258
  • Cannot be merged as of too many conflicts. Partly resolved in PR #2258 with conflicts conservatively resolved #2318 and I found the standalone PRs for some of those changes as well, which we should merge preferably then.
• Rebase and merge https://github.com/motioneye-project/motioneye.eo
  • A you did already start, so it's now python3-intl branch 🙂.

Approximately from simple to difficult, regarding possible conflicts 😉.

[starbasessd]

I would like to continue contributing in my way (docs and support, rather than code) if you'd like, as a collaborator.

[MichaIng]

I just invited you to the organisation. This way you should be able to create branches right here on the main repo, but I think this was already the case before, since you were a collaborator already.

[starbasessd]

I don't know anything about programming, especially in the github environment, I was able to edit the wiki documents, and assist end users as needed, and marking issues with the various flags (which hadn't been done prior to my being given collaborator status. I am a script-kiddie in python2 & 3, pretty good with bash and dos command (win) scripting, and so forth. jasaw (video/motion issues) and jawsper (brought an image of motionEyeOS) are both very good in what they do. I am a professional beta tester, too, since back in the Zoom Modem voicemail days (for the C64 & Early PCs), I can break things quite easily. I am an Enterprise (Level 2 & 3) Senior Help Desk, for the past 25+ years. I do translations between end users and developers. I have a lot of resources available for testing (VBox VM Server Ryzen7 CPU 32GB RAM/10TB Storage, Gbit network, Pi1, 2, 3B, 3B+, 4B-4GB and 4B-8GB, PiZeroW, 8 Network IP cams, 2 USB Cams, 1 CSI PiCam) and used them regularly for helping out here. I do docker some, too. If you go through the WiKi, I've done a lot of updates to a lot of the pages. If someone uses an esoteric (read: non-Debian based) OS, I'm willing to try to get it to work, and then document it...
Let me know if you can continue to use my 'services'....

[starbasessd]

One other question: Will this be applied to motionEyeOS too, or no?

[MichaIng]

That is very valuable, always good to have someone with some focus on the docs, issues/bug report review and testing side. Also Docker experience is good, which I lack, while I see the Dockerfile here.

I haven't had a look into motionEyeOS yet. I personally would concentrate on getting motionEye itself up and ready for Python 3 first, and then update Docker images and motionEyeOS in a second step. But those two definitely need others with more insights and experience with Docker and BuildRoot.

[kleini]

I would like to contribute, too, if my spare time allows me to do that. I am a long time developer and familiar with Python 3, Docker, Kubernetes.

[cclauss]

My preference would be to add #2308 as step 0. given that it is now green and puts in place some important guardrails.

[MichaIng]

@cclauss
Okay, it doesn't hurt anyway. I made some comments as minor enhancements, otherwise good to be merged from my end.

I just invited you both as members to the orga. I'll send out invitations to all motionEye collaborators, which have their permissions preserved anyway. That way we also see who of those is still active and interested to continue contributing to the project 🙂.

[jmichault]

Hello,
my python3-intl branch is ready to be merged : #2313

[MichaIng]

Awesome, many thanks for your work. From my end, I'd merge python3 first into dev and then rebase and merge PRs with python3 as base onto dev. It practically may not make any difference, after I resolved conflicts between those branches with the result that python is now up-to-date with dev, but it somehow feels more natural to me 😅.

[jmichault]

@kleini,
Hello, I tried to modify Dockerfile to take into account all the changes made for python3 and internationalization.
Can you take a look to see what's left to do?

[MichaIng]

From what I see it contains all changes which were applied to dev/python3 branches before, so should be fine. Only thing which went inside as well is that python3-pip is not purged for the final image, while python-pip was purged before. If I'm not mistaken, motionEye does not require pip, so to reduce the container size a bit, this could be re-added to the list of purged packages. But as always, also since you switched (reasonably!) to Bullseye, it simply needs to be tested 🙂.

[kleini]

Just had a quick look on the "Install in Docker" wiki page and the Dockerfile. Building the container image works and the container works. Getting a picture from my network cameras works as well. So for a first step everything is fine. General questions from my side would be:

• Where is CI gone? I would like to build container images in CI and publish to docker hub or into one of the other container image registries.
• Today I would not build applications inside containers any more. I would build the outside and copy over the build artifacts into the container to reduce container size. But looking at the Dockerfile, a small container size was not a primary target before. Instead the container image should be able to do anything.
• Initial configuration file /etc/motioneye/motion.conf does not work.
• Help for configuration options does not work.
• Is there some Python 3 base container in some registry? Would that be the better base for a container image?
• How does motionEye "talk" to motion? I think, it would be usefull to separate motionEye und motion into separate containers if communication is possible across container boundaries.
• Move away from Docker and use buildah, podman, cri-o, containerd to get rid of no real open source dependencies.

[cclauss]

Where is CI gone?

https://github.com/motioneye-project/motioneye/tree/dev/.github/workflows

[starbasessd]

Initial configuration file /etc/motioneye/motion.conf does not work.

Because motion is controlled by motionEye, and the motion.conf settings needed are in /etc/motioneye/motioneye.conf

[starbasessd]

I think, it would be usefull to separate motionEye und motion into separate containers if communication is possible across container boundaries.

Why? Why increase the complexity? Multiple points of failure? Diagnostics issues when something does break? Different versions / builds / new dependencies when something is updated? Have one force update the other? When motion updates and breaks the 'current' version of motionEye (it has happened) how do you control either?
With fixed versions in a single container, you update as a package.
The Docker IMO brings everything into the package needed to run to assist with the above questions.

[starbasessd]

From what I see it contains all changes which were applied to dev/python3 branches before, so should be fine. Only thing which went inside as well is that python3-pip is not purged for the final image, while python-pip was purged before. If I'm not mistaken, motionEye does not require pip, so to reduce the container size a bit, this could be re-added to the list of purged packages. But as always, also since you switched (reasonably!) to Bullseye, it simply needs to be tested 🙂.

pip is purged (and the ability to exec in the container) to prevent people from making changes (similar to the RO file systems in motionEyeOS), from what I've discovered. Pretty effective. motionEye doesn't need pip after installed. Pip is only needed after install in a 'normal' install to do an upgrade:

pip install motioneye --upgrade

[kleini]

I think, it would be usefull to separate motionEye und motion into separate containers if communication is possible across container boundaries.

Why? Why increase the complexity? Multiple points of failure? Diagnostics issues when something does break? Different versions / builds / new dependencies when something is updated? Have one force update the other? When motion updates and breaks the 'current' version of motionEye (it has happened) how do you control either? With fixed versions in a single container, you update as a package. The Docker IMO brings everything into the package needed to run to assist with the above questions.

Because this large container needs to be rebuild completely for every little change. Maintaining a deployment is much easier, if different tools - motionEye and motion in this case - are in separate containers. The correct combination of containers can be controlled with various tools like docker-compose, kubernetes, Helm charts.
And furthermore motion requires a lot of CPU resources. Having those split across multiple containers improves overall performance. An ideal case would be to have every process running in its own container.
A container based on some normal OS like Debian Buster in this case, is often a problem with containers. Containers should not run an OS, making those containers error prone and vulnerable. As previously stated, a container just runs a single process. Having tools separated in this way, allows to have a motionEye container image, which only contains some Python runtime and motionEye, nothing else. If motionEye has any vulnerabilities the underlying container allows nearly no operations except running motionEye.

An incompatibility problem between motionEye and motion is a problem of pinning versions of both tools. This is not solved by a single container image. This must be solved by pinning these versions and it does not matter, whether they are in the same container image or in separated ones. And the motionproject already does provide a container image, so we don't need to download some built motion any more and include that in our container image.

[starbasessd]

Are you going to support cross host setups too, when people using clusters install motion on one host, and motionEye on another host?

[kleini]

Are you going to support cross host setups too, when people using clusters install motion on one host, and motionEye on another host?

This would be an interesting approach.

[cclauss]

Because this large container needs to be rebuilt completely for every little change.

This is not really true. There is tons of documentation about the order of commands in a Dockerfile to reduce long builds.

[starbasessd]

The advantage, IMO, of a complete container, is you don't care what the docker host is running. The same docker for motionEye runs on all the debians, the RHs, the Arch, whatever. (Theoretically, Windows, too) without consideration of what is needed vs what is already installed on the host, and no communications issues between the packages (if the sandbox system WORKED, they would not be able to easily talk to each other, or possibly other containers...)

[starbasessd]

Because this large container needs to be rebuilt completely for every little change.

This is not really true. There is tons of documentation about the order of commands in a Dockerfile to reduce long builds.

If you read the option 3 in Install In Docker it pulls existing base 'images' of the OS, installs only what motionEye needs, and doesn't need 'get a cup of coffee' break. This was modified because of the age of software included in the existing version, and documented in the spirit of open source / open documentation on how to 'make your own mods'. If it goes the way of 'single process per container, and the complexities of managing many (possibly many, many?) packages and containers, I may fork the project (I will for me) because I deal with average newby non-programmers and developers who want to learn how to make changes or fit their situation without the need to go through a 4 year IS college degree to understand it.

[jowax]

Hi, I'd like to contribute. Is it possible to add me as a collaborator?

[MichaIng]

Many thanks for considering to contribute to motionEye. Before it becomes to inflationary, IMO good practice is to add you to the contributors list, after your did some contribution, and not beforehand 🙂. I'm all in for being open and inclusive, but a minimum level of trust building is fine I think.

[avanc]

Development should be cooperated with thingOS, since motionEyeOS is based on this project.

I fully support this, as I'm using thingOS for photOS :-)
Thus, it would be great if improvements to the base software would go upstream (i.e. merged into thingOS).

I might be able to support on the base system. Just @mention me.

[avanc]

A considerable computing power is needed and the whole process takes a few hours.

There bigger the argument to use GitHub actions runners for this slightly_smiling_face. They don't have much CPU power, so it might even take longer, but no own hardware is blocked see_no_evil.

@MichaIng Feel free to have a look at https://github.com/avanc/photOS/tree/master/.github/workflows
I'm building photOS for 4 platforms (RaspberyPi 1-4) in parallel based on git tags.
But there might be potential for optimization. Sometimes I hit the GuthubActions time limit...

[MichaIng]

You mean to merge improvements from photOS into thingOS? I suggest to open PRs in this case.

However, I'm the wrong person to discuss about thingOS 😉.

[avanc]

My intention was to use the photOS Github workflow as starting point for automatic compilation of motioneyos :-)

[avanc]

Ah, in with my first comment I meant to port improvements from motioneyeos back to thingOS, so photOS can also benefit from it ;-)

[strips]

Any thought about switching Motion to Motionplus?

I just gave up on Motioneye and Motion for my new Rasperry Pi Camera V3. I have not got any of the workarounds to work. Motionplus works but the GUI is lacking. Switching to Motionplus will give good support for libcamera.

[NuclearPhoenixx]

Any thought about switching Motion to Motionplus?

I just gave up on Motioneye and Motion for my new Rasperry Pi Camera V3. I have not got any of the workarounds to work. Motionplus works but the GUI is lacking. Switching to Motionplus will give good support for libcamera.

Yeah good luck with that. I think you should very much look forward to the Python 3 port at the moment, that's the most important thing right now AFAIK.

[bigmac5753]

is there an ETA on the next release?
seems to be taking forever

[MichaIng]

Not really. To speed things up, would be great if someone could write release notes. Then I'll merge #2675 to push a pre-release to PyPI.

[localsnet]

If I correct understand, that now migration on Python3 is actually nothing happening (just a translation to other languages) ?

[MichaIng]

Could you rephrase your question? I do not understand.

Multi-language support is probably the most visible change aside of the Python 2 to Python 3 migration. But there were a lot of other changes and fixes, especially around the upload options and methods. Check out the list of merged PRs with the milestone assigned: https://github.com/motioneye-project/motioneye/pulls?q=is%3Apr+is%3Aclosed+milestone%3Av0.43.0

[localsnet]

@MichaIng , you answered on my question actually, this is what i meant, thank you.
Do you have documentation on CI/CD part or it's CI files only?
Thanks.

[va1entin]

@MichaIng Happy to see that motioneye lives on and gets the well deserved Python 3 migration! Thanks for your work everyone 😃

With a new team at the helm and new features already added or in the works, I was wondering if there was any interest in this functionality I requested to merge five years ago now: #821
At the time Calin didn't want to merge it because it adds pynacl as a new dependency and he expected most users to not be interested in encrypting media files uploaded to the cloud by motioneye or using separate tooling for that.
If you as the team are interested, please let me know and I'll gladly make a new PR. I've been using the functionality for 5 years now and still think it's a nice feature :)

Aside from that, are there any technical blockers remaining for the 0.43.0 release or is the list of issues assigned to the milestone representative of all the work left? Maybe I can help with something.

[beppo-dd]

@cclauss I wrote you an email. Can you invite me to your organsiation?

[cclauss]

@beppo-dd You have created no issues and no pull requests. Why should you be named as a project maintainer?
https://github.com/motioneye-project/motioneye/issues/created_by/beppo-dd
https://github.com/motioneye-project/motioneye/pulls?q=is%3Apr+author%3Abeppo-dd

[beppo-dd]

@cclauss This was of course not my intention.
My question arose because I wanted to correct a wiki page in motioneyeos, but I couldn't find any ways to edit it: In Github, there is no way to do pull requests on other wiki pages, see #846. I can clone the wiki repository, but I have no right to write on it, see this answer of the stackoverflow question "On GitHub, can I fork … a wiki".
Question: Can I get a possibility to correct the wiki page that belongs to the motioneyeos reposity?

[cclauss]

Now I understand. Let's try this approach.
Let's say that you want to make edits to wiki/Manual-Download-And-Installation

• Make a new pull request with the file wiki-edits/Manual-Download-And-Installation.md
• Your first commit should be exactly the same text as the wiki file but with two modifications to the filepath.
  • Add -edits to the directory name.
  • Add .md as the file name suffix.
• Your second (and later) commits should be your desired changes.
• This pull request will never be merged but if a maintainer finds the modifications useful then they may choose to transfer them to the corresponding wiki page.

Leaving the first commit be the wiki page unmodified will allow maintainers to spot exactly what was changed.

I hope this process makes sense.

[beppo-dd]

@cclauss I hope, I did it right: see motioneye-project/motioneyeos/pull/2998.
Please be sure to read my "Warning" at the end of the pull request description.

[theGiuMac]

Good morning,
Do you still need help in coding, git ops (mergin, rebasing...) and so on?
Specifically on the migration to Py3, I might be of help. Anyhow, any news on its release?

[cclauss]

@theGiuMac Python 2 died 1,372 days ago on 1/1/2020.

It is therefore a shame that the default branch of this repo still points to a Python 2 implementation.

If you are interested in this functionality, please look at the dev branch instead. It is 800+ commits ahead of the default.

• https://github.com/motioneye-project/motioneye/tree/dev

[theGiuMac]

@cclauss thank you for the prompt answer. The state of development is clearer now. And indeed, "Sit tibi terra levis" Python 2.

I probably wrote my comment in the wrong place, I've found the link in the homepage. I was looking for info specifically on motionEyeOS or if I could help in anything related to the coding side.

I guess you're keeping issues open about current problems and I've read that DevOps seems in place (at least for the CI/CD pipeline).

[cclauss]

Can the default branch please be changed to something more modern than Python 2?

[McMagnus]

I 2nd this. I've been digging into motion/motioneye since last year, haha, but just now did I realize there's a (working?) Python3 implementation.

What could be the downside of using it as default?

I can help on the dev side if needed. And I'll try that dev branch right now.

[nullr0ute]

There is now a beta release with a move to python3 here https://github.com/motioneye-project/motioneye/releases/tag/0.43.1b1

With luck that should be a stable release soon. Testing I suspect is the most important thing :)

[cclauss]

With luck that should be a stable release soon.

In our lifetimes? Python 2 died 1,463 days ago on 1/1/2020.

[MichaIng]

I thought to keep a branch as default which contains a stable release, which is currently python2 or master only. But I agree in the meantime, the Python 3 compatible beta version should work much better than the old release. I'll create a main branch from beta and make this the new default.