Using Entitas in generic way, instead of generators

[WeslomPo]

Sorry that's off topic, what is the advantage of using generators in particular, instead, for example, generic methods or barehands? Why there no any of tutorial, how to use entitas without generators
notimetoexplain - use generators? Lack of time to make some, I think.

Today I spend whole day trying to work with Entitas and Generators, but it always broke all my code when I remove some parameters from components, or rename it. And that is annoying. 1 minute to change component, 30 minutes to generate code somehow... And there only TWO COMPONENTS. >___<. How it work when there ten of them, thousands, how it use with other teammates? Yeah, I know about compilers, reflections, mono, roslyn and etc, but it is so hard to work with(especially on mac), I did not even try to use them because of complexity.

I start thinking about two solution, first one, copy components in their generated environment (in different namespace, or in class), and use in system only components from there. It prevents of broke code and interrupt compiling process (in theory). Because it very hard to implement with one evening with my knowledge about entitas, I found second solution, that I'm prefer:
Get some generated classes, and make them in generic way. It took about three hours and I make some working solution to start working with Entitas without generator.

Here are a my three hours code, maybe need to add some methods...
Context:

namespace Generic {
// IInitializable is Zenject interface
	public class Context<TEntity> : Entitas.Context<TEntity>, IInitializable where TEntity : class, IEntity, new() {
		public readonly LookupTable Table;
		public readonly Matcher<TEntity> Matcher;

		public Context(string name, LookupTable table, Matcher<TEntity> matcher)
			: base(table.Size, 0, new ContextInfo(name, table.Names, table.Components)) {
			Table = table;
			Matcher = matcher;
		}

		public void Initialize() {
#if(!ENTITAS_DISABLE_VISUAL_DEBUGGING && UNITY_EDITOR)
			if (!UnityEngine.Application.isPlaying)
				return;
			var observer = new ContextObserver(this);
			UnityEngine.Object.DontDestroyOnLoad(observer.gameObject);
#endif
		}

		public TEntity Get<T>() where T : IComponent {
			return GetGroup(Matcher.For<T>()).GetSingleEntity();
		}

		public bool Has<T>() where T : IComponent {
			return Get<T>() != null;
		}

		public TEntity Set<T>(T component) where T : IComponent {
			if (Has<T>())
				throw new Exception("Something went wrong with setting " + typeof(T) + " component.");
			TEntity entity = CreateEntity();
			entity.AddComponent(Table.IndexOf<T>(), component);
			return entity;
		}

		public TEntity Replace<T>(T component) where T : IComponent {
			TEntity entity = Get<T>();
			if (entity == null)
				return Set(component);
			entity.ReplaceComponent(Table.IndexOf<T>(), component);
			return entity;
		}

		public void Remove<T>() where T : IComponent {
			DestroyEntity(Get<T>());
		}

	}
}

LookupTable:

namespace Generic {
	public class LookupTable {

		public Type[] Components { get; private set; }
		public string[] Names { get; private set; }

		public int Size {
			get { return Components.Length; }
		}

		public LookupTable(Type[] components) {
			Components = components;
			Names = new string[components.Length];
			for (int i = 0; i < components.Length; i++) {
				Names[i] = components[i].ToString();
			}
		}

		public int IndexOf<T>() where T : IComponent {
			// Todo assert if -1
			return Array.IndexOf(Components, typeof(T));
		}

	}
}

Matcher

namespace Generic {
	public class Matcher<TEntity> where TEntity : class, IEntity, new() {

		private readonly LookupTable _table;
		private readonly Dictionary<Type, IMatcher<TEntity>> _cache = new Dictionary<Type, IMatcher<TEntity>>();

		public Matcher(LookupTable table) {
			_table = table;
		}

		public IMatcher<TEntity> For<T>() where T : IComponent {
			Type type = typeof(T);
			if (_cache.ContainsKey(type))
				return _cache[type];
			var matcher = (Entitas.Matcher<TEntity>) Entitas.Matcher<TEntity>.AllOf(_table.IndexOf<T>());
			matcher.componentNames = _table.Names;
			_cache[type] = matcher;
			return _cache[type];
		}
	}
}

Entity

namespace Generic {
	public class Entity : Entitas.Entity {}
}

How to use

1. Create a context class:

namespace Generic {
	public class GameContext : Context<Entity> {
		public GameContext(LookupTable table) : base("Game", table, new Matcher<Entity>(table)) {

		}
	}
}

2. Install it with zenject, or just make by your hands and save in singletone (that's bad)

namespace Installers {
	public class ContextInstaller : MonoInstaller {
		public override void InstallBindings() {
			GameContext();
		}

		public void GameContext() {
			Type[] components = {
				typeof(DebugMessageComponent),
				typeof(PositionComponent)
			};
			Container.Bind<LookupTable>().FromInstance(new LookupTable(components)).WhenInjectedInto<GameContext>();
			Container.BindInterfacesAndSelfTo<GameContext>().AsSingle();
		}
	}
public class ConextInstallerNoDependency {

		private static GameContext _game;
		public static GameContext Game {
			get {
				if (_game == null) {
					Type[] components = {
						typeof(DebugMessageComponent),
						typeof(PositionComponent)
					};
					_game = new GameContext(new LookupTable(components));
				}
				return _game;
			}
		}

	}
}

3. Use it

namespace Systems {
	public class DebugMessageSystem : ReactiveSystem<Entity>
	{
		private readonly GameContext _context;

		public DebugMessageSystem(GameContext context) : base(context) {
			_context = context;
		}

		protected override Collector<Entity> GetTrigger(IContext<Entity> context) {
			return context.CreateCollector((context as GameContext).Matcher.For<DebugMessageComponent>());
		}

		protected override bool Filter(Entity entity) {
			return entity.HasComponent(_context.Table.IndexOf<DebugMessageComponent>());
		}

		protected override void Execute(List<Entity> entities)
		{
			foreach (var e in entities) {
				DebugMessageComponent component = (DebugMessageComponent)e.GetComponent(_context.Table.IndexOf<DebugMessageComponent>());
				// This is my bicycle instead of Debug.Log
				// D.Log(component.Message);
				Debug.Log(component.Message);
			}
		}
	}
	public class HelloWorldSystem : IInitializeSystem, IExecuteSystem {
		private readonly GameContext _context;
		// I do not love to new objects in update, that's why I cached it, but this is not necessary 
		private DebugMessageComponent component;

		public HelloWorldSystem(GameContext context) {
			_context = context;
		}

		public void Initialize() {
			component = new DebugMessageComponent {Message = "Test" + Random.Range(0, 1000)};
		}

		public void Execute() {
			component.Message = "Message " + Random.Range(0, 1000);
			_context.Replace(component);
		}
	}
	public class Hello : Feature {
		public Hello(GameContext context) : base("HelloFeature") {
			Add(new DebugMessageSystem(context));
			Add(new HelloWorldSystem(context));
		}
	}
	// This class may be MonoBehaviour, but I use Zenject and prefer simple classes
	public class GameController : ITickable, IInitializable, IDisposable {
		private readonly GameContext _context;
		private Entitas.Systems _systems;

		[Inject] // Zenject attribute for injection
		public GameController(GameContext context) {
			_context = context;
		}

		// No injection
		public GameController() {
			_context = ConextInstallerNoDependency.Game;
		}
  		// OnEnable
		public void Initialize() {
			_systems = new Feature("Systems").Add(new Hello(_context));
			_systems.Initialize();
		}
		// Update
		public void Tick() {
			_systems.Execute();
			_systems.Cleanup();
		}
		// OnDisable
		public void Dispose() { }
	}
}

Gist

[danielbe1]

If you don't like the code generators you can take a look at [Entitas-Lang(https://github.com/mzaks/ECS-Lang)

BTW it appears [here](https://github.com/sschmid/Entitas-CSharp/wiki/Tools-and-Extensions in the docs in case you've missed it, there are a couple more tools there you might be interested in.

[FNGgames]

I know it's not exactly the point of your post, but you can re-gen your code in a few short steps which takes me about 1 minute. I do this if I make any breaking changes to a component.

1. Move Components, Systems and GameController.cs to a folder outside your project (e.g. desktop).
2. Delete the contents of your Generated folder
3. Regenerate contexts (ctrl-shift-G)
4. Move Components folder back
5. Regenerate components (ctrl-shift-G)
6. Move Systems and GameController.cs back and you're done.

It's not the robust solution you're looking for, but it should definitely not take you 30 mins to re-gen if something breaks.

[WeslomPo]

@FNGgames unity have a folder WebplayerTemplate, you can move files here if you don't want to compile. Your algorithm is very clear (and I can make a script to make it automagical), but on start, before of understand how framework is working exactly, it's make beginners feel very uncomfortable. I want to say this is huge problem in a core of framework.

@danielbe1 I dig trough all wiki, I think. I don't understand why I need other tool to work with Entitas, and I don't like Eclipse :). Maybe, someday I will try it, but first, let me make a bicycle :). Maybe it will be working with generators some day, maybe I will not need it ever.

[WeslomPo]

There a two problem:
1. We have complex dependency model
2. IEntity/Entity - is premature optimized with generators, and a lot of code dependent on it.

Complex dependency model.
Systems, and a lot of code dependent from generated code and model, generated code dependent from model components too, when we made change on model, we need to update all other dependencies. There are an image with arrows that represent dependencies (#paintskilz).

Here a systems dependent from components bypass generated code. Maybe, if we freeze model and make generated code and systems dependent from it, we can solve the problem of egg and chicken. I mean, we can just copy/paste components and made dependencies from it instead of source model, that's brake cross generated code dependency in systems, and solve compiler errors, I think. Here another image:

No dependencies from source components. No compiler errors when you change them.

Premature optimization
That is bad. Maybe you not need a code generator at all, anyway 95% of frame time eaten by graphics. Maybe need to concentrate on building more comfortable workflow with developers, that can save a lot time for making a lot of new features, than on optimization that save 1% of frame time, and cost a hundreds of developers hours.

[sschmid]

Hi. I understand that the current workflow with the reflection-base code generator might be a pain, especially in the beginning when you're new to Entitas and code changes all the time. Good news is that we're currently working on finding better solutions like Entitas-lang or a roslyn-based code generator which will work even when there are compile errors.
The code generation issues will fade more into the background once your code gets more stable. I'm in a team with 4 devs, we have over 300 components and over 600 systems. Code generation is actually no problem. But I get that it's a problem currently. The roslyn solution should fix all of this.

On premature optimization:
Using "premature optimization" when talking about Entitas is wrong IMHO. I have very strict rules when writing libraries such as Entitas. I always strive for simplicity and efficiency. Nothing gets added without being proved to be necessary. I do sth I call Profiler-Driven-Development and all the changes and improvements are actually backed by numbers and tests.

Code generation was a natural evolution of the library and necessary to improve performance and optimized memory usage. As it turns out, it also enables us to do automatic object pooling and results in a great api.
In the early days we had an api like this

entity.AddComponent(new PositionComponent());
var pos = entity.GetComponent<PositionComponent)();

There is no object pooling of components and retrieving components like this was very slow and created a measurable bottle neck.
Code generation enabled us to do automatic object pooling for components and speeded up component lookup by an order of magnitude. At the same time we get a nice api.

Debug.Log(e.debugMessage.message);

// vs

DebugMessageComponent component = (DebugMessageComponent)e.GetComponent(_context.Table.IndexOf<DebugMessageComponent>());
Debug.Log(component.Message);

I really like this api, that's why I went with code generation. Luckily code generation is optional and flexible. You can chose to only use a small subset of generator like ComponentsLookup or nothing at all.

But your code suggestions contain a few nice ideas that I also wanted to try when tackling #307

[sschmid]

That is bad. Maybe you not need a code generator at all, anyway 95% of frame time eaten by graphics. Maybe need to concentrate on building more comfortable workflow with developers, that can save a lot time for making a lot of new features, than on optimization that save 1% of frame time, and cost a hundreds of developers hours

A multiplayer backend doesn't do anything involving graphics and it's a question of real money if you need 1000 servers or if you can do the same with 100 servers with an optimized framework

[sschmid]

I think a really good point you made is the dependency image. I'll think about it

[WeslomPo]

@sschmid yeah, I understand about multiplayer and etc. But beginners may not get there because of complicity of work with generators.

Is Roslyn work with mac?

I made wrong implementation at first, now I move all get/set component code in Entity that extends base class, and i remove LookUpTable because it just ContextInfo. Now it looks like this:
DebugMessage component = e.GetComponent<DebugMessage>();

LookUpTable now just statically extends ContextInfo methods:

public static class LookupTable {

		public static int IndexOf<T>(this ContextInfo contextInfo) where T : IComponent {
			// Todo assert if -1
			return Array.IndexOf(contextInfo.componentTypes, typeof(T));
		}

		public static int[] IndicesOf(this ContextInfo contextInfo, Type[] types) {
			int[] indices = new int[types.Length];
			for (int index = 0; index < types.Length; index++)
				indices[index] = Array.IndexOf(contextInfo.componentTypes, types[index]);
			return indices;
		}

	}

Generic.Entity:

namespace Generic {
	public class Entity : Entitas.Entity {

		public void AddComponent<T>(T component) where T : IComponent {
			AddComponent(contextInfo.IndexOf<T>(), component);
		}

		public void RemoveComponent<T>() where T : IComponent {
			RemoveComponent(contextInfo.IndexOf<T>());
		}

		public void ReplaceComponent<T>(T component) where T : IComponent {
			ReplaceComponent(contextInfo.IndexOf<T>(), component);
		}

		public T GetComponent<T>() where T : IComponent {
			return (T) base.GetComponent(contextInfo.IndexOf<T>());
		}

		public bool HasComponent<T>() where T : IComponent {
			return HasComponent(contextInfo.IndexOf<T>());
		}

		public bool HasComponents(Type[] components) {
			return HasComponents(contextInfo.IndicesOf(components));
		}

		public bool HasAnyComponent(Type[] components) {
			return HasAnyComponent(contextInfo.IndicesOf(components));
		}

		// Not all methods here, but, maybe it not useful in real workflow.

	}
}

I updated gist with full implementation. I think, that code some "not written now" generator of code can convert to more fast in a particular case, if necessary.

Conclusion

I understand that there a framework with long history, with robust workflow and etc. I just want to give a few ideas, show an example of work without a code generator. Thanks for answers!

Gist with full implementation

[danielbe1]

Correct me if I am wrong, but under that solution, the code might compile, but might fail at runtime if any mistake is done. The code generated version will not compile and not fail at runtime and is much more IDE friendly.

[WeslomPo]

@danielbe1 as a generic solution, it will not compile if you made some type conversion mistakes, as it guarantee type safe.
But, that code can have some cpu and cache problem, that lay more deep down than type checking (as all logical errors). Every call is a search a type in array, and that may lead to cpu down, when you use many of components and many of matchers. Replacing components with "new" every time make a new object and replace previous, that need to be collected by garbage collector sometimes, that makes a hiccups in cpu page in profiler here are @sschmid and @mzaks talk more. There need to think about caching and pooling. It not hard as described in video, but it still need work to do with. I think it still good performance, as It work with fast Entitas inside, only user code may have problems... I hope... :)

Also, to be honest, I don't test my solution not for errors, not for speed or efficiency, I made it for learning. I want to start using ECS as fast as I can, without fighting with compiler, that I hate to do. If I would love to fight with compiler I'm will programming on C++ xD.

Code generated solution more robust, more user friendly and very performant solution, but it have defect with compiling while you in active developing, and made many of changes in components, like remove, rename fields and etc. I love to remove and rename something every time :) and that defect makes me crazy, I feel unconfident and uncomfortable.

[Spy-Shifty]

Renaming is quite easy. Use your IDE for that job (Visual Studio or something else).
Deleting and Adding Fields can be indeed a problem. But it don't take hours as you described before.
The real problem (at the moment) are blueprints. If you delete a field of a component wich you use in a blueprint, you have to recreate the hole blueprint... (because of the binary serialization). Thats my real problem by the way.

I had my difficulties with the codegen at beginning, too. But now i feel comfortable with it. Sure, there could be some improvements. But speed is very important!
I build a game with thousands of "bots" and still I have 75 FPS. That's inconceivable with the "intuitive" Unity way (Monobehaviour Scripts attached to each bot)

The best way to learn and use entitas is to take a look at the examples and videos. And first think about components and then act! And finally, the simplest solution is the right solution ;)

[WeslomPo]

@Spy-Shifty there problem that when you rename field (with resharper, of course) or change signature, it need to be regenerated, but it can't because of reflection.
Now I thinking about my diagram, that I can make by myself with barehands, to test that solution, maybe that will be enough to solve problem, maybe not. After tests I will write result here. But now I have no time to experiments, today and tomorrow I need to release three games simultaneously xD.

Generic solution, to my opinion will works same as generated solution, maybe with suspectible lack of performance, but still fast than MonoBehaviour, because in core it is a entitas with their awesomness.

Thanks for your advices.

[WeslomPo]

@sschmid, Back to my first message. I said that I have two solutions, and seconds is using generic wrapper, that wraps entitas with generic methods. But first solution is make a copies of components and make dependencies to that copies instead of source components, I suggested, that this solution will fix issue with impossibility of re-generating when you change source component a lot. But, also I suggested that solution will need a lot of time to work... but last turned out it is not true :)

I spend two hours and made some peglegs in code generators to test my suggestion. And it works. It really breaks that vicious circle of dependencies. Now I can use power of generators, and make a changes without pain in ass :).

A few comments, that solution very simple, but need to make several changes in code generators, and that solution generate only simple components with fields only (it not copy-paste file, it use arguments list to made component). It need change all dependencies in Systems to generated components. I made it like a proof-of-concept, and it works.

[KumoKairo]

That second dependency graph which you've shown describes exactly what's Entitas-Lang achieves. It stores Components as separate files, while generated code is that Component Snapshot. I have some ideas about visual component editor (based on uFrame), which will allow the same thing (effectively storing components as editor assets, leaving the generated code to be Component Snapshot). I don't take any steps towards that at the moment because there's so much to do with tutorial videos and blueprints

[WeslomPo]

@KumoKairo yes, you right. But the fact is, that even with coupled with project compiler (like mono in unity) you can re-generate components after changing. I test that and that's works (I told about in chat). Third-party program not needed.

[KumoKairo]

Can you post some relative performance charts? Make a few use cases like Execute System with GetComponent, ReplaceComponent etc. With, say, 1000 entities. It will provide a great value to this topic in general.

[yosadchyi]

Quick draft of generic API: https://github.com/yosadchyi/Entitas.Generic

[mic-code]

How does Entitas-Lang work with player modding? It appears to me that a generic api having the ability to load arbitrary component at runtime is key to easy player modding, and modding capability is major reason to use a ECS system(instead of unity normal way) to me

[leonhardrocha-zz]

@WeslomPo has really strong points. I don´t see any future for Entitas if this is not fixed asap. I don´t have to spend 30 min trying to fix errors just because I renamed some component... and the code becomes too cluttered... I guess this is the opposite direction to go!

[dreadicon]

I have to agree on the modding issue; I intend for my game to be very mod-friendly. To that end, I want my environment to be as close to what modders will use as possible, and using the unity editor and/or hard-compiled generated code seems to run contrary to that. IMHO, moddability is one of the most appealing aspects to ECS, so it's a shame Entitas doesn't play as nicely with runtime modification.

[BenjaBobs]

Is this still being discussed? I personally think it would be a huge improvement in developer experience, especially since it would remove a whole step from the pipeline. Maybe some more bench marking would shed some light on whether this is feasible. @yosadchyi's results are certainly promising.

[Deepscorn]

@BenjaBobs, @sschmid I checked @yosadchyi's results and implemented my own.
The reason is I wanted pooling and don't like writing "new" each time.

entity.Replace(new Position(position.Location + velocity.Linear * dt,
                    position.Rotation + velocity.Angular * dt));

I implemented through generics + lookup table here https://github.com/Deepscorn/EntitasWithoutCodeGeneration Looks nice to me. What do you think?