Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

not an issue. just some suggestions #8

Open
aferust opened this issue Apr 9, 2024 · 38 comments
Open

not an issue. just some suggestions #8

aferust opened this issue Apr 9, 2024 · 38 comments

Comments

@aferust
Copy link

aferust commented Apr 9, 2024

Will you implement all from scratch? Or you can speed up using existing libraries

I suggest you to borrow code around

more for nogc
https://github.com/AuburnSounds/Dplug/blob/master/core/dplug/core/nogc.d
https://github.com/AuburnSounds/Dplug/blob/master/core/dplug/core/thread.d
http://mir-algorithm.libmir.org/ mir libraries provide betterc or nogc alternatives for std.algorithm

@al1-ce
Copy link
Owner

al1-ce commented Apr 9, 2024

Thanks for suggestion and links!

It'll greatly help me build everything up

Mostly my goals for this library is to build somehwat of a nogc runtime and if I can borrow more efficient code I will (with right authorship notices ofc)

@al1-ce
Copy link
Owner

al1-ce commented Apr 9, 2024

Actually biggest problem is typeinfo
I have no idea how to do it properly besides of what __traits provides

@aferust
Copy link
Author

aferust commented Apr 9, 2024

I never attempted to implement it or need it actually.
This search lists somethings
https://github.com/search?q=org%3Alibmir+typeinfo+language%3AD&type=code&l=D

@aferust
Copy link
Author

aferust commented Apr 9, 2024

İf you inherit all extern c++ classes from a extern c++ CppObject, you may achieve something similar maybe

@al1-ce
Copy link
Owner

al1-ce commented Apr 9, 2024

https://github.com/libmir/mir-algorithm/blob/f3d0b7fe4cb0e4a52c8929a1ac85213f5e123bd3/source/mir/type_info.d

Hmm
Not sure how it works
Also I saw typeid which uses D's TypeInfo which is nono, though it's for some pointer things..
Dunno dunno

@aferust
Copy link
Author

aferust commented Apr 9, 2024

@al1-ce
Copy link
Owner

al1-ce commented Apr 9, 2024

I think I'll pass on that one

@al1-ce
Copy link
Owner

al1-ce commented Apr 9, 2024

Oh, while you're here
Just thinking
Would it be good idea changing name of package to something like stdcpp or stdc
Problem with that is it might be confusing with core.stdc and core.stdcpp
Edit: maybe clib or cpplib
Honestly kinda confused on all that naming thing rn

Edit:
Had a crazy idea to have all of modules top level and name them something like vector.h (i.e import vector.h;). Bad idea, but technically can be done

@aferust
Copy link
Author

aferust commented Apr 9, 2024

I am not good at naming stuff either. I should reflect your intention. maybe "stld". not sure though.

@al1-ce
Copy link
Owner

al1-ce commented Apr 9, 2024

If to reflect intension then it'd be something like nogcstd
Maybe I'll go with clib. It's simple, consise and nice to type
Gonna do some reogranisation today then and some changes in readme

@aferust
Copy link
Author

aferust commented Apr 9, 2024

i think this is doable:

https://github.com/dlang/dmd/blob/master/druntime/src/object.d#L587

create a nogc port of TypeInfo class as extern(C++) class TypeInfo https://github.com/dlang/dmd/blob/master/druntime/src/object.d#L587

you will also need porting extern(C++) class TypeInfo_Class : TypeInfo https://github.com/dlang/dmd/blob/master/druntime/src/object.d#L1575

then all extern(C++) classes must have the required member methods via a template mixin or deriving them from a CPPObject class having the required methods. (this is how D classes actually work, all of them is implicitly derived from an Object class)
https://github.com/dlang/dmd/blob/master/druntime/src/object.d#L110

Then create a function:
TypeInfo_Class _typeid(T)(T cppclass) ;
TypeInfo_Class _typeid(T)() ;

toString method must yield: modulename.ClassName // traits does the trick name field is important to compare two instances or two types, see here: https://github.com/dlang/dmd/blob/master/druntime/src/object.d#L848

opEquals, getHash, equals, compare, name, etc.

You are doing this because you want to provide runtime type information for the situations:

pseoudo code:

import core.stdc.stdio;

extern (C++) class Base {}
extern (C++) class Derived : Base {}

extern(C)
int main() {
Base base = _new!Derived();

// Check the actual runtime type of the object
if (_typeid(base) == _typeid!(Derived)) {

// _typeid(Derived)) access name field via traits, this have to be a compile time variable that may be produced with traits
// _typeid(base) access name field at runtime. it should not be difficult after figuring out _typeid!(Derived))
    printf("base is of type Derived");
} else {
    printf("base is not of type Derived");
}

return 0;
}

@al1-ce
Copy link
Owner

al1-ce commented Apr 9, 2024

This might work
I'll try tomorrow

@al1-ce
Copy link
Owner

al1-ce commented Apr 10, 2024

Ok, typeinfo is impossible for now
DMD complains about missing symbols

/bin/ld: /home/al1-ce/.dub/cache/clib/~master/build/library-debug-OqusfGrASdkkikmE3QPpNA/libclib.a(classes_5_39a.o):(.data._D4clib7classes9cppObject7__ClassZ+0x0): undefined reference to `_D14TypeInfo_Class6__vtblZ'
/bin/ld: /home/al1-ce/.dub/cache/clib/~master/build/library-debug-OqusfGrASdkkikmE3QPpNA/libclib.a(classes_5_39a.o):(.text.d_dso_init[.data.d_dso_rec]+0x22): undefined reference to `_d_dso_registry'
collect2: error: ld returned 1 exit status

And LDC complains about missing symbols

/bin/ld: /home/al1-ce/.dub/cache/clib/~master/build/library-debug-1Sw9Nsl7VdWbAC49c21ZkQ/libclib.a(clib.classes.o):(.data._D4clib7classes9cppObject7__ClassZ+0x0): undefined reference to `_D14TypeInfo_Class6__vtblZ'
collect2: error: ld returned 1 exit status

All that it seems because classes are in libraries and they have no typeinfo which brokenD still expects for some god unknown reason

Is what I was going to say, but turns out I had to compile library with -betterC flag
Got something working I guess

@aferust
Copy link
Author

aferust commented Apr 10, 2024

Are you really sure to go for betterC? Even c++ has a runtime. You don't have to go betterC for nogc nothrow code.

I would invest nogc code over betterC. İ believe it is not a reasonable approach to expect full functioning OO features without a runtime. C and betterC is already there for the particular needs. What I and many needs in d is a standard library without gc. Please read this article

https://www.auburnsounds.com/blog/2016-11-10_Running-D-without-its-runtime.html

Not all runtime use GC.

I made DCV nogc nothrow. And i am happy with it. https://github.com/libmir/dcv.

@aferust
Copy link
Author

aferust commented Apr 10, 2024

Please also note that reference counted class objects will not be possible untill Walter Bright implements some compiler internals. Then you can forget shared pointers like in c++ for class objects

@al1-ce
Copy link
Owner

al1-ce commented Apr 10, 2024

Are you really sure to go for betterC? Even c++ has a runtime. You don't have to go betterC for nogc nothrow code.

I'm just accounting for it

I would invest nogc code over betterC.

Yea, nogc all the way

What I and many needs in d is a standard library without gc

That's what I'm doing plus wierd typeinfo thing I did today and will not touch soon

Please read this article

Ooh, I'll note that

Please also note that reference counted class objects

Not sure what you're talking about

I'm just doing things I can to maybe get some easier time in future, also like, if it's -betterC compatible then it's nogc compatible you know

@aferust
Copy link
Author

aferust commented Apr 10, 2024

@al1-ce
Copy link
Owner

al1-ce commented Apr 10, 2024

To be completely honest I turn on -betterC flag to completely shut up GC

It's extremely leaky and Valgrind constantly complains about those leaks which distracts me from preventing my own leaks

Yea...

Edit:
Just tried using extern(C) __gshared string[] rt_options = [ "gcopt=disable:1" ]; and import core.memory; GC.disable();. Dumb was I

Edit2:
Still there's a loss of around 72 bytes coz 23106. Only betterC silences that

@al1-ce
Copy link
Owner

al1-ce commented Apr 11, 2024

Ok, spent a lot of time, but now I can do this:

interface IInterface {}
class Parent: CppObject {}
class Child: Parent, IInterface {
    mixin RTTI!Parent;
    mixin RTTI!IInterface;
}

_typeid!Parent.isBaseOf!Child; // true
_typeid!CppObject.isBaseOf!Child; // true
_typeid!IInterface.isBaseOf!Child; // true
_typeid!CppObject.isBaseOf!IInterface; // false
_typeid!CppObject == _typeid!Parent; // false
Child c;
_typeid(c).name; // gonna give fully qualified name of type

And so on, something pointer, something name

The THING is that it's so complicated and it can't give even a fraction of information of D's TypeInfo
Not sure I enjoy doing that and keeping everything GCless. I'm gonna leave type_info as it is now, if you gonna have any better implementations then my one then PRs are welcome

@al1-ce
Copy link
Owner

al1-ce commented Apr 11, 2024

Ok, turns out I'm way overthinking it
_new thing works on D classes as well as on extern(C++) classes. Same for free
And D TypeInfo works even without GC
So... this TypeInfo thing is only for BetterC now, which means mostly screw it

@aferust
Copy link
Author

aferust commented Apr 11, 2024

I will try your typeinfo. I am mostly offline now because of holiday. İt is not so easy but maybe i run my little evalex library with your classes https://github.com/aferust/evalex

@al1-ce
Copy link
Owner

al1-ce commented Apr 11, 2024

Honestly if to not delve into brokenD then you can just do this:

import clib.memory;

class DClass {}

void main() @nogc {
    DClass d = _new!DClass();
    TypeInfo t = typeid(d);
    _free(d);
} 

And it WILL work I've tested it!

https://github.com/al1-ce/clib/blob/master/test/nogc/test_nogc.d

@aferust
Copy link
Author

aferust commented Apr 12, 2024

I found a little moment to test typeinfo and i get two serious linker errors on Windows with ldc. One is related to d array copy. I have a workaround for it i will make a PR for it. Other is about virtual table. vrtbl stuff.

@al1-ce
Copy link
Owner

al1-ce commented Apr 12, 2024

Can you send those errors here?

@aferust
Copy link
Author

aferust commented Apr 12, 2024

for the first one, we need this. This is a known bug ldc-developers/ldc#2425

version(LDC) extern(C) void _d_array_slice_copy(void* dst, size_t dstlen, void* src, size_t srclen, size_t elemsz){
import ldc.intrinsics;
llvm_memcpy!size_t(dst, src, dstlen * elemsz, 0);
}

the second one:

clib.lib(clib.typeinfo.obj) : error LNK2001: unresolved external symbol _D14TypeInfo_Class6__vtblZ

@al1-ce
Copy link
Owner

al1-ce commented Apr 12, 2024

clib.lib(clib.typeinfo.obj) : error LNK2001: unresolved external symbol _D14TypeInfo_Class6__vtblZ

It should't use any. Maybe you're trying to use betterC version in non-betterC or vise-versa?
I though I put some safety in there

@aferust
Copy link
Author

aferust commented Apr 12, 2024

İn the test code,'s dub.json i have dflag betterC

@al1-ce
Copy link
Owner

al1-ce commented Apr 12, 2024

No idea then. Gonna try building in LDC, but can't say about windows since I'm on linux

@al1-ce
Copy link
Owner

al1-ce commented Apr 12, 2024

Wait a minute

Is dependancy clib:betterc or just clib?

Its important distinction
If you not using clib:betterc its gonna reference D's TypeInfo coz its how it works

@aferust
Copy link
Author

aferust commented Apr 12, 2024

i also corrected to use subpackage. still I had to make some changes to clear some compiler errors.

in memory.d

void _free(T)(ref T t) @nogc nothrow {
    // If there's ~this we wanna call it
    static if (__traits(hasMember, T, "__xdtor")){ 
        assumeNothrowNoGC(
        (T x)
        {
            return x.__xdtor(); // without this compiler complains about __xdtor is not nogc
        })(t);
        
    }
    import core.memory : pureFree;
    pureFree(cast(void*) t);
    // And if T is nullable then make it null
    static if (__traits(compiles, { t = null; })) t = null;
}

import std.traits;
auto assumeNothrowNoGC(T) (T t) if (isFunctionPointer!T || isDelegate!T)
{
    enum attrs = functionAttributes!T
               | FunctionAttribute.nogc
               | FunctionAttribute.nothrow_;
    return cast(SetFunctionAttributes!(T, functionLinkage!T, attrs)) t;
}

/// Unmanaged allocator (a wrapper over malloc)
    extern(C++) class Mallocator(T): CppObject, IAllocator!T if (isValidAlloccatorType!T()) { // typo cppObject ?
        mixin RTTI!(IAllocator!T); // need T here. 

.....


And i had to remove alignment parameter from all occurences of allocate, reallocate etc.

my dub.json:

{
	"name": "test",
	"dependencies": {
		"clib:betterc": {"path": "clib-master"}
	},
	"dflags": ["-betterC"]
}

test code is your test_betterc.d

@al1-ce
Copy link
Owner

al1-ce commented Apr 12, 2024

Alignment parameter kinda has to be there since it'd be required for more advanced allocators
Gotta find a way to preserve it

Xdtor should be nogc nothrow if ~this is nogc nothrow, gotta be ldc error

@aferust
Copy link
Author

aferust commented Apr 12, 2024

and this:

char[200] __cpp_class_inheritance_generator(T)() {
import core.stdc.string: strlen;
char[200] s = ' ';
char[] t = cast(char[]) __traits(fullyQualifiedName, T);

s[0..7] = cast(char[]) "void __";

for (int i = 0 ; i < t.length; ++i) {
    if (t[i] == '.') {
        s[7 + i] = '_';
    } else {
        s[7 + i] = t[i];
    }
}

s[(7 + t.length)..(7 + t.length + 18)] = "_clib_parent (){}\0";

foreach (ref c; s)
{
    if(c == '!') // a workaround for template class or intefaces
        c = '_';

}
return s;

}

@al1-ce
Copy link
Owner

al1-ce commented Apr 12, 2024

Yea that one I already discovered and fixed
Though by checking if char is not alpha or underscore

@al1-ce
Copy link
Owner

al1-ce commented Apr 13, 2024

Ran this abomination:

test_complete:
    dub build
    dub build --compiler ldc2
    dub build :betterc
    dub build :betterc --compiler ldc2
    dub test
    dub test --compiler ldc2
    dub test
    dub test :betterc --compiler ldc2
    dub run :test_nogc
    dub run :test_nogc --compiler ldc2
    dub run :test_betterc
    dub run :test_betterc --compiler ldc2

LDC does complain about _d_array_slice_copy, your code for that function fixed it all and there's no problem on my machine after that.

I'm gonna roll out a new version of lib when I'm done with lists and maps

@al1-ce
Copy link
Owner

al1-ce commented Apr 14, 2024

I finally understand why there's no nogc implementation for at least some basic STL

Making it is like throwing myself at wall

> making containers
> should container free data on dtor
> i guess
> segfault
> have to make a lot of ammends to ensure that containers don't segfault on each other
> containers need error handing for out of bounds and fail malloc
> exceptions don't work
> make my own exceptions
> how to catch
> found janky way to catch
> exceptions need to print where they failed
> have to either make everything that can fail a template or get stack trace
> templates ugly
> spend day trying to get proper backtrace
> function names are magled
> d demangler doesn't work
> have to write my own
> my optional type doesn't work with containers because they're freeing themselves 
> fine, will not use them
> segfault
> segfault
> segfault
> something freeing itself when it shouldn't 
> segfault
> screw demangling
> backtrace doesn't work with dmd
> aboout to give up

@aferust
Copy link
Author

aferust commented Apr 14, 2024

Please always remember d classes use reference semantics, and extern(C++) classes too. You should use structs for the containers to make life easier.

Nogc exceptions exist.
https://github.com/libmir/dcv/blob/26a82d425449991c2a10d6ed3c80171957cf88d2/source/dcv/imgproc/filter.d#L148

Another way allocate your exception with _new and deallocate it in your catch block. dplug library should have something like that.

there are module literals FILE and LINE maybe they help locating exception.

https://github.com/dlang/DIPs/blob/master/DIPs/other/DIP1008.md

Please also note that in nogc code you can even run GC code with debug label, of course only for debug builds

debug throw new Exception( ...
debug writeln...

@al1-ce
Copy link
Owner

al1-ce commented Apr 14, 2024

Containers are structs, that's the problem and I personally refuse to make them classes because imo if something is like a basic type or container it should be struct and if it's more complex then it's class
Plus I'd have to duplicate everything as normal class and as extern c++ class because betterc

LINE and FILE give you only immediate line and file
Example:

0 struct vector {
1     fail() {
2         msg("fail");
3     } 
4 }
5
6 vector v; v.fail();
7 
8 void msg(string F = __FILE_, int L = __LINE__) (string msg) {
9    writeln(F, " ", L, " ", msg);
10 } 

Will print test.d 2 fail instead of test.d 6 fail

To fix that I'd have to pass line and file everywhere which is bloat

Nogc exceptions I don't remember what was about them but there was something

Might've been betterc

Might drop betterc actually

Gonna do testing, so much testing. Hmm, maybe for solving segfaulting self-destruct I could do something like rust's borrow checker with this(this) ctor (just found out about it, cool thing)

@al1-ce
Copy link
Owner

al1-ce commented Apr 15, 2024

I have invented RefCounting!

Putting it here as a reminder

import std.stdio;
import core.stdc.stdlib;

void main(string[ ] args) {
    test!int a = {42};
    test!char b = {'b'};
    test!bool c = {false};
    
    munk(a);
    func(a);
    nunk(a);
    
    junk();
    punk(255);
    punk(100);
    
    test!string d = sunk();
    
    munk(b);
    func(c);
    
    writeln("eof");
}

test!int ti;

test!string sunk() {
    test!string t = {"nope"};
    t = test!string("here here here");
    return t;
}

void junk() {
    test!string t = {"test"};
    func(t);
}

void punk(int k) {
    test!int t = {k};
    ti = t;
}

void munk(T)(test!T t) {
    func(t);
}

void nunk(T)(test!T t) {
    ti = t;
}

void func(T)(test!T t){}

struct test(T) {
    T t;
    size_t* ctr = null;
    
    this(ref scope test!T o) {
        if (o.ctr is null) {
            o.ctr = cast(size_t*) malloc(size_t.sizeof);
            if (o.ctr is null) writeln("malloc failed");
            o.ctr[0] = 1;
        }
        ctr = o.ctr;
        t = o.t;
        ctr[0] = ctr[0] + 1;
    }
    
    ~this() {
        writeln("dtor for ", t);
        if (ctr is null) return;
        ctr[0] = ctr[0] - 1;
        writeln("ctr - ", ctr[0], " of ", t);
        if (ctr[0] == 0) {
           writeln("delete ", t);
           free(cast(void*) ctr);
        } 
    }
}

The trick is that counter is a pointer, which keeps it synced across instances

And surprisingly it all works kinda flawlessly. Gonna have to test it in real environment though

Gone will be ages of constant ref, scope and segfaults

Well, if that works and if issue with exceptions will be just betterc (for which i will abandon betterc) then it's way better, yes

Edit:
Just realised one "downside" with this approach. Everything will have to be pointers. In case of vector data is already pointer, but capacity and size will have to become pointers too (not sure about allocator tho). Not too big of a problem, but a thing to consider nontheless

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants