error: name 'bytearray' is not defined

[setop]

I seems Codon does not understand bytes/bytearray/ByteString.

I tried to type with ByteString (from typing module) but is just makes thing worst.

What is the good way to write programs that manipulate bytes ?

[arshajii]

Hi @setop, note that Codon's strings are (currently) ASCII, so you can often just use a str in place of an immutable byte array. For a mutable byte array, you could use List[byte] (Codon will store this as a contiguous array in memory). We'll likely add bytearray in a future version.

[setop]

Thanks

How can I transform a List[byte] into something that can be written to disk. Here I get error: 'File' object has no method 'write' with arguments (File, List[byte])

[learnforpractice]

Use pickle module as the example shown below:

import gzip
import pickle

def read_test():
	path = 'testjar.bin'
	jar = gzopen(path, 'rb')
	a = pickle.load(jar, List[byte])
	print(a)
	jar.close()

def write_test():
	path = 'testjar.bin'
	jar = gzip.open(path, 'wb')
	a = [byte(1), byte(2), byte(3)]
	pickle.dump(a, jar)
	jar.close()

write_test()
read_test()

[setop]

I'm trying to implement a codec whom format has already been specified. So I cannot change it to pickle+gzip. I have to be able to handle raw bytes. Hence the initial question : "What is the good way to write programs that manipulate bytes ?".

[learnforpractice]

Solutions:

• Write a parser yourself if you know the codec format
• Port a python library to codon.
• Use a python library directly, as shown in these docs: https://docs.exaloop.io/codon/interoperability/python

[setop]

I found an hidden method in File implementation,

codon/stdlib/internal/file.codon

Line 43 in fc70c83

def __file_write_gen__(self, g: Generator[T], T: type):

.

codon run <<< 'import sys;sys.stdout.__file_write_gen__([byte(65),byte(66)])' produce AB.

But I'm not sure if it should be used ...

[setop]

Solutions:

• Port a python library to codon.

I already have an implementation in Python, using bytearray.
My point in re-implementing it in Codon is to make is faster.

• Use a python library directly, as shown in these docs: https://docs.exaloop.io/codon/interoperability/python

Executing python code in Codon will make it pointless (see #178).

[learnforpractice]

If your goal is to optimize performance while making minimal modifications, there are alternative solutions available that you can consider:

1. Use mypy
2. Use pypy
3. Use cython
4. Use taichi
5. Use numba
6. Use pythran

Indeed, there may be other solutions available, but these are the ones that I am aware of at this time.

[arshajii]

Thanks

How can I transform a List[byte] into something that can be written to disk. Here I get error: 'File' object has no method 'write' with arguments (File, List[byte])

You could use ''.join(chr(int(b)) for b in v) where v is the list of bytes. Then you can just file.write() that string. Let me know if this works in your case.

[setop]

If your goal is to optimize performance while making minimal modifications, there are alternative solutions available that you can consider:

1. Use mypy
2. Use pypy
3. Use cython
4. Use taichi
5. Use numba
6. Use pythran

Indeed, there may be other solutions available, but these are the ones that I am aware of at this time.

Mypy is a type checker, the compiler is mypyc. There is also Nuitka. But they mostly do like Cython : melt a python interpreter into c code and call a C compiler to build an executable. This does not produce faster programs.

Pypy is a JIT Compiler. It does not produce standalone executable. It is a drop-in replacement of the official interpreter. It is sometimes surprisingly faster and sometimes surprisingly slower than the official interpreter. In best case don't expect more than 10x.

Numba, Pythran and Taichi are specialized in scientific computing, using dedicated annotations.

And there is Codon that can sometime achieve 100x faster. And that's good news, I'm here to evaluate it.

I've been successful in the past at writing code that run on both python interpreter and Codon. Sometimes it is only a matter at changing a type or a structure.

So, if you don't mind, I'll try to pursue ;-)

[learnforpractice]

Yeah, nice to meet you here! I just provided you with some suggestions to see if they align with your needs. Definitely, Codon is amazing, and it's a good idea to keep exploring its features.

[inumanag]

This is something that we will need to implement in our standard library. No ETA for now 😞, however we'll be happy to accept PRs for this feature.

[inumanag]

Merging into #608 .