.Net: Replace KernelBuilder with Kernel.CreateBuilder

[stephentoub]

@matthewbolanos, @markwallace-microsoft, entirely up to you whether you want to merge or close this; fine with me either way, I'm just opening this to make it easy to decide. Fixes #4033.

This removes KernelBuilder and adds a static CreateBuilder() such that instead of doing:

Kernel kernel = new KernelBuilder().Whatever().Build();

one would now do:

Kernel kernel = Kernel.CreateBuilder().Whatever().Build();

Everything else should be functionally the same.

[markwallace-microsoft]

LGTM @matthewbolanos do you agree?

[qihangnet]

What is the purpose of making such destructive changes repeatedly during the RC phase?
Do you have any misconceptions about RC?

[markwallace-microsoft]

What is the purpose of making such destructive changes repeatedly during the RC phase? Do you have any misconceptions about RC?

Hi @qihangnet
apologies for the additional breaking changes. This one was made after an internal review of the API surface and was deemed important for consistency with other .Net API's. The team is working very hard to make sure we provide the best experience for developers. We are on track to have a stable 1.0 API very soon and end the cycle of breaking changes.

[strikene]

What is the purpose of making such destructive changes repeatedly during the RC phase? Do you have any misconceptions about RC?

Hi @qihangnet apologies for the additional breaking changes. This one was made after an internal review of the API surface and was deemed important for consistency with other .Net API's. The team is working very hard to make sure we provide the best experience for developers. We are on track to have a stable 1.0 API very soon and end the cycle of breaking changes.

After I upgraded from RC3 to RC4, I couldn't even build the kernel properly
CS7069 Reference to type 'IKernelBuilder' claims it is defined in 'Microsoft.SemanticKernel.Core', but it could not be found

I kind of wonder if this kind of massive breaking change at this stage of RC4 means that more RC is needed before it can be officially released, and we spent quite a bit of time going from Beta 8 to RC3, even though RC3 still has stack overflows that lead to having to fall back to Beta 8

[stephentoub]

What is the purpose of making such destructive changes repeatedly during the RC phase? Do you have any misconceptions about RC?

Hi @qihangnet apologies for the additional breaking changes. This one was made after an internal review of the API surface and was deemed important for consistency with other .Net API's. The team is working very hard to make sure we provide the best experience for developers. We are on track to have a stable 1.0 API very soon and end the cycle of breaking changes.

After I upgraded from RC3 to RC4, I couldn't even build the kernel properly CS7069 Reference to type 'IKernelBuilder' claims it is defined in 'Microsoft.SemanticKernel.Core', but it could not be found

It sounds like something in your solution is still pulling in the RC3 version, such that some assemblies think the interface is defined in the abstractions assembly and some think it's defined in the core assembly.

[qihangnet]

@strikene, With the release of RC4, there have been significant changes to the namespace and package names related to Connectors. I suggest confirming if you are using the latest version. In my experience, the use of outdated RC3 packages is often the reason for this situation. PR #4230 provides a comprehensive overview of these updates.

I also shared your concern about the significant changes in naming conventions without sufficient documentation or guidance. It seems that there is a difference in understanding between the Semantic Kernel team and the more conservative changes we expect at this stage of the “Release Candidate” (RC); the extent of these changes feels more like the Beta stage rather than the RC stage.

So far, almost all the community-related articles, blogs, video, including official Microsoft documents, have become obsolete, making the experiences of all relevant developers outdated. This reminds me of some things from the Windows Phone era. Many in the community are very concerned about these arbitrary changes, even angry, and some have lost patience and turned to LangChain.

Unfortunately, it looks like these renaming and disruptive changes will continue to occur. This could significantly damage SK's efforts to engage with the community.

[markwallace-microsoft]

Unfortunately, it looks like these renaming and disruptive changes will continue to occur. This could significantly damage SK's efforts to engage with the community.

There will be no further disruptions in the API surface once we reach v1.0.0 (in the next few days). Our goal is to provide a stable Semantic Kernel that our partners and internal teams can rely on.

[madsbolaris]

I want to plus one what Mark has shared. The last round of changes during our RC were made explicitly because of feedback from the community and from the .NET SDK experts at Microsoft. Has we lock on v1.0 next week, all non-experimental APIs will not change.

I also want to share that we were able to republish all of the content in learn.Microsoft.com to reflect the latest in RC4. We expect few (if any) breaking changes from now and until V1.0 next week

[qihangnet]

@markwallace-microsoft @matthewbolanos Thank you very much for your timely reply. I respect your work and dedication. Regardless of weekdays or weekends, I am fortunate to witness the relentless efforts of your team.

I initiated and led the Chinese Semantic Kernel Technology Community, which currently has nearly 1000 members. Tomorrow, at the conference called “.NET Conf China 2023” in Beijing, we have arranged nearly ten lectures on AI and Semantic Kernel. However, the recent frequent disruptive changes have brought challenges to our community and triggered some negative reactions. We have advised everyone through articles and WeChat groups to update their projects after the official release of version 1.0, to avoid these changes causing significant negative impacts repeatedly. But, this also means that any information conveyed to the Semantic Kernel audience during this time may become outdated at any time, which is obviously not a pleasant experience for the participants. Based on these considerations and our doubts about the value of this PR, we are providing feedback to you.

I believe that perhaps the SK team should try a relatively conservative approach—marking the old methods as deprecated before introducing new methods, instead of immediately removing them. Progressive changes are usually more acceptable and understandable than disruptive changes.

Once again, thank you for taking the time to address my concerns amidst your busy schedule. If my words have been offensive in any way, please accept my sincere apology. I sincerely hope that the Semantic Kernel project can grow steadily and rapidly.

[qihangnet]

Oh,one more thing. Recently, the access level of some practical classes in the Semantic Kernel has been changed from public to internal. This change has affected some developers who used these classes and methods directly before. Due to this change, developers are now forced to extract and duplicate the necessary code from the source code of the Semantic Kernel repo, or create equivalent functionality on their own. If your schedule permits, I request that you evaluate this issue and consider possible solutions before the release of version 1.0. Thank you again.🤭

[stephentoub]

the access level of some practical classes in the Semantic Kernel has been changed from public to internal. This change has affected some developers who used these classes and methods directly before.

It'd be helpful for future reference to know which types/members you're referring to and how they were being used / the scenario.

In the incubation of SK, there have been many experiments, many approaches tried, etc., and as is commonly the case with such efforts, that left a lot of things in an inconsistent state, many things that weren't necessarily still useful, etc. When preparing for a stable 1.0 release, one really needs to remove as much of that as possible, as once you're stable, you can effectively never remove anything, since you're signing up to avoid breaking changes. Since it's always possible to add later but effectively never possible to remove later, the general philosophy is to get rid of things that aren't deemed valuable, as they can be brought back later if proven otherwise.