Synchronize topic replicas + partitions

[jrevillard]

Hello,

This PR fixes #34 + also manage replicas.

For topic config modifications and deletion also I added in output the previous value in order to have more information (I can update the way it is displayed if you want).

Best,
Jerome

[devshawn]

Thanks for the PR! I will review it this weekend.

[jrevillard]

Any news @devshawn ?

Best,
Jerome

[devshawn]

Hi @jrevillard, sorry I have been out sick lately. I am adding a few tests and releasing a new version shortly; if you could rebase on that I will take a look and get this merged in this week. Thank you!!

[jrevillard]

Hi @devshawn , rebase done.

Best.

[devshawn]

Awesome. Once again, apologies for the delay. I am going to review, run some additional tests against cloud clusters (e.g. confluent cloud), and then if all looks good, merge it in and get a release out. I don't have time until the weekend but I'm finally free for a weekend and will happily get this taken care of.

Thanks again @jrevillard!

[jrevillard]

Any news @devshawn ?

Best.

[devshawn]

Apologies again! Taking a look at this and will merge it in this weekend.

[devshawn]

I've been testing it around and it seems to be working for all situations -- I'll probably do a pre-releases/snapshot of this first since its such a large change.

[jrevillard]

Hello,

Any news about it please?

I wonder what are your plans for kafka-gitopt ? do you still plan to develop it or not ? I see a couple of very interesting missing features that can be done but if you think that you might not have any more time to develop it I might look for other solutions...
Perhaps sharing the repository management with the community could help ?

Best,
Jerome

[sarahgudc]

Hi,

I am also curious about the future roadmap of kafka-gitops. It's a really interesting tool and I would like to adopt it into a project assuming there are ongoing plans for development. I also would find this PR a very useful feature! :)

thanks very much,

Sarah

[devshawn]

I do plan to further develop kafka-gitops as well as take in PRs from anyone :)

I had a crazy past few months which prevented me from having time to work on projects. I have made time this weekend to review and merge this PR.

[devshawn]

All of my testing so far has been looking good. Should finally be released by the end of the weekend :)

[MartinBurian]

@devshawn I'm sorry to look like I'm pushing, but I find this tool to be the simplest gitops solution for kafka management and we're sorely missing this configuration option 😞 Can I help in any way to get it done? Otherwise, I would need to create a fork of kafka-gitops with this merged for our use, or, in the worst case, use Terraform for the job...

[jrevillard]

@devshawn I'm sorry to look like I'm pushing, but I find this tool to be the simplest gitops solution for kafka management and we're sorely missing this configuration option disappointed Can I help in any way to get it done? Otherwise, I would need to create a fork of kafka-gitops with this merged for our use, or, in the worst case, use Terraform for the job...

Hello @MartinBurian , you can use my fork if you like. I also plan on the coming weeks to integrate the work of @Twb3 concerning schema registry and add JSON support.

[casimcdaniels]

@devshawn So, are we merging this or not? Please open up this project to more contributors if you're backlogged.

[devshawn]

Hi @jrevillard @casimcdaniels @MartinBurian, my apologies on the lack of support on the project. I had quite a backlog of other projects over the past few months.

I will review and merge all the pending PRs and get the latest updates pushed by the end of this weekend, I promise this time -- I have booked my calendar accordingly. We can look into opening this up to more collaborators after cleaning up the current list of PRs and issues. Thank you everyone for the contributions!!

Look forward to a release by the end of this weekend, although I plan to get to it sooner than that.

[devshawn]

Doing some more testing and hope to get a release out shortly :)

[jrevillard]

Still no news @devshawn ?

[devshawn]

@jrevillard Sorry I actually caught a case of influenza that's been keeping be down this week. I am still working on getting this out within the next week.

[JianTray]

We would also be interested in getting this PR deployed? Would be very useful for tracking our topics :)

[devshawn]

Pre-release build will be released shortly.

[JianTray]

There's an error in line 126 of KafkaService.java which is causing an index out of bounds error. The line should read:
replicas.remove(replicas.size() - 1);

Also getting an error with mBeam:

javax.management.InstanceAlreadyExistsException: kafka.admin.client:type=app-info,id=kafka-gitops
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:64)
	at org.apache.kafka.clients.admin.KafkaAdminClient.<init>(KafkaAdminClient.java:500)
	at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:444)
	at org.apache.kafka.clients.admin.Admin.create(Admin.java:69)
	at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:49)
	at com.devshawn.kafka.gitops.service.KafkaService.buildAdminClient(KafkaService.java:195)
	at com.devshawn.kafka.gitops.service.KafkaService.getTopicDescription(KafkaService.java:173)```

And the -command-config argument seems to do nothing atm :)

[irisgve]

Hi @devshawn, this particular merge doesn't seem to be included in any release yet. The latest release seems to be from Feb 2021. Are there any plans of creating a release for all the recent changes including this one? Thanks.

[dal13002]

@devshawn I believe the latest image release (ie on dockerhub) doesn't have this change. Any plan to release a new version?

[iNviNho]

@devshawn Two years have passed and this is still not released.

You should really consider giving more people rights to manage this repository.