<phil_bb>
But that went... about as well as can be expected
luna-is-here has quit [Ping timeout: 248 seconds]
livoreno has joined #commonlisp
akoana has left #commonlisp [#commonlisp]
akoana has joined #commonlisp
admich1 has quit [Ping timeout: 244 seconds]
admich1 has joined #commonlisp
<aeth>
Any chance for a bilingual (only when allocated specially so you can get a fixed pointer) heterogeneous data structure that can represent structs and arrays-of-structs (the same pattern repeated n times) with all of the type work in the type prefix like an array? Even if it's just bit, the C-compatible ([un]signed-byte *), single-float, double-float, and maybe foreign-pointer...
<gilberth>
Why don't you just make one?
<aeth>
Wouldn't it need implementation support or a heroic amount of hacks (across different implementations)?
<aeth>
static-vectors exists, but (simple-array foo (*))s exist for some values of foo so you just use AREF
akoana has left #commonlisp [#commonlisp]
<aeth>
And some things like (unsigned-byte 64) and double-float are normally boxed, but not always, so how do you handle that? Implementations may optimize (simple-array double-float (*)) and (simple-array (unsigned-byte 64) (*)) in certain contexts, but this isn't one
akoana has joined #commonlisp
<gilberth>
Depends. If you can live with an (simple-array (unsigned-byte 8) (*)) to be your object, all is fine. If you want a custom header saying that is some binary struct, you would perhaps need to teach the GC that there's immediate data and no pointers.
<aeth>
well, (unsigned-byte 32) or (unsigned-byte 64) is probably easier to work with if you're mostly seeing single-/double-float and int32/uint32/int64/uint64, although you'd still need to worry about endianness a bit when splitting/merging
<aeth>
and float-features:single-float-bits (and bits-single-float, double-float-bits, bits-double-float, etc.) or low level CFFI-adjacent hacks if you want to mix integers and floats
<aeth>
And if you're mixing integers and floats, you'd have to decide "is this mostly floats?" (encode the CL integers as CL floats; the FFI side won't care) or "is this mostly integers?" (encode the CL floats as CL integers; again, the FFI side won't care)
akoana has left #commonlisp [#commonlisp]
<aeth>
By contrast, the only thought put into static-vectors is that you have to free it (on some implementations, which means that to be portable you have to free it). It's a pointer for CFFI and a simple-array for CL
<aeth>
So the existence of static-vectors makes CFFI structs a million times more annoying.
<gilberth>
aeth: My idea is to just use (THE (SIMPLE-ARRAY <what you need>) STORAGE-VECTOR) with (SAFETY 0). Doesn't matter what the actual array type says. When in doubt use the FFI or SAP primitives.
<gilberth>
And then have a macro to define your accessors according to the layout you specify.
akoana has joined #commonlisp
<gilberth>
What I want to smuggle into CCL with time is the reverse of static-vector. Given any foreign pointer make it mimick as a Lisp array. That can be done by placing array headers beneath the heap at some low address and make a displaced arrays to those. That would be handy if what you want to access is allocated under control of that foreign lib.
<gilberth>
For freeing static vectores you could install finalizers. But that's unreliable. You would start piling up foreign data when you don't cons much Lisp data. Happened to me with Free-CLIM. There really is no good solution for that.
<gilberth>
Here is another implementation idea. In Closure I had the problem that the Lisp only had 7-bit characters and I wanted a string datatype fit for Unicode (16-bit at that time) and so (simple-array (unsigned-byte 16) (*)) was the choice. I called that new string data type a ROD. But: I wanted them to print like #"foo" and being able to have a RODP. I solved that by putting all those arrays into a weak hash table to tell.
<aeth>
Most properly written C stuff (which probably isn't most C stuff) or externed-to-C C++ stuff wants you to bring your own allocator these days.
<aeth>
I think.
<aeth>
If the interfaces are standard enough, you could probably hook it into the implementation and finally make finalizers work :-p
<gilberth>
Good. Then static-vectors would be all you need.
brokkoli_origin has quit [Ping timeout: 252 seconds]
jeffrey has quit [Quit: jeffrey]
<aeth>
yes, because if you're giving a buffer instead of data you can just make it a buffer from static-vectors
<gilberth>
But yes, many C libs want that you bring your own buffers.
<aeth>
the problem is (safety 0) type punning is less-safe-than-C (because the compiler won't catch it, or if it can, you have to trick it, too)
brokkoli_origin has joined #commonlisp
<aeth>
a library _may_ be implemented exactly the same way (on implementations where (safety 0) breaks everything), but at least then you won't have to think about it and all bugs can be fixed upstream (at least in principle)
<gilberth>
aeth: Think practical. You would implement it by making a struct defining macro. It's up to the accessors to do the type check.
<aeth>
type checking is slow
<aeth>
when it's implementation-supported, it can safely be removed on slot writing if it infers or is DECLAREd, and when you read it, it can feed into similar inference (which you have no access to), which can simplify having to deal with multiple such structs or arrays-of-structs
<gilberth>
aeth: Then make up your mind. Can't have both safety and no type checks.
<gilberth>
Have you checked if you can put immediate data into DEFSTRUCT with SBCL? back-to-back? Maybe that's possible.
rkazak has quit [Ping timeout: 252 seconds]
luna-is-here has joined #commonlisp
<aeth>
the more general solution would be to have an "array-of-struct" sort of data structure that is kind of like a defstruct, but only stores data that it knows isn't heap allocated... sort of like how simple-arrays tend to work
<aeth>
(Padding is simple, just make padding slots that you don't set to.)
<gilberth>
If you want that this mirrors struct { double x; float y; char c; } you would still need to follow the relevant ABI.
<aeth>
I suspect structs must have an adjacent prefix so they can't go back to back, unless you can hack it into the padding.
<aeth>
which, although they waste space, could work for sending structs to the GPU, since you'd have control over what the structs are (and can insert padding)
<gilberth>
No, you can't place the contents of structs back to back. My question was that when I say (defstruct foo ((x :type double-float) (y :double-float))) and would look at the memory I'd find a double at 'x' and not a pointer to a Lisp double.
<aeth>
gilberth: As for the ABI, I don't think so. As long as it works (the construct and a way to pin it if you're FFIing it), it works.
<aeth>
Everything else can be built on with macros
<gilberth>
aeth: There might be surprises when going from SYSV to Windows or from AMD64 to ARM64. Just saying.
<aeth>
yes, you'd have to #+foo it, etc.
cow_2001 has quit [Quit: ✡]
<aeth>
But if you can define a compact heterogeneous structure, possibly repeated, with the ability to pin a pointer to it and get that pointer if requested, then you can build the rest with metaprogramming, I think?
<aeth>
relying on even defstruct double-floats to exist compact seems not particularly useful. Especially because it might only work on SBCL, for some versions, on some platforms, with no guarantee that it won't regress, etc.
<aeth>
Plus, you'd have to "unroll" any arrays (if small enough) into repeated slots like x-1 y-1 z-1 x-2 y-2 z-2
cow_2001 has joined #commonlisp
puke has joined #commonlisp
<aeth>
I suppose, as I just described, you'd still have to unroll any nexted arrays-in-structs or structs-in-structs since you can only say something like (double-float single-float (unsigned-byte 32)) and not something like (double-float (simple-array single-float (3)) (unsigned-byte 32))
<aeth>
s/nexted/nested/
<aeth>
I suppose one level of simple-array nesting as I demonstrated for the supported numeric types would support the most common cases
<aeth>
so foo would be something like (foo (double-float (simple-array single-float (3)) (unsigned-byte 32)) 2) with, optionally, a pinnable pointer
rkazak has joined #commonlisp
louis77 has quit [Quit: Client closed]
leeb_ has joined #commonlisp
leeb has quit [Ping timeout: 252 seconds]
shawnw has quit [Ping timeout: 265 seconds]
josrr has quit [Ping timeout: 252 seconds]
akoana has quit [Quit: leaving]
istewart has joined #commonlisp
<fe[nl]ix>
gilberth: what you mentioned (a simple-array displaced to a foreign pointer), is within the API of static-vectors
<fe[nl]ix>
Allegro and LW support that by multiplexing the keyarg :allocation on cl:make-array
random-nick has quit [Ping timeout: 260 seconds]
<fe[nl]ix>
it's necessary when you have foreign APIs that manage the I/O buffers, like liburing or libusb (IIRC)
livoreno has quit [Quit: Connection closed for inactivity]
<aeth>
fe[nl]ix: which gives me the idea that maybe the simplest approach is to just always use static-vectors even for (simple) structs and expect compilers optimize around the inline functions in float-features as well as various bit operations
<aeth>
Then it stays (mostly) plain CL with a wrapper layer on top of the "struct" arrays and a wrapper layer beneath for the compiler optimizations
<aeth>
i.e. (defun foo (a) (declare (type (simple-array single-float (3)) a)) (float-features:single-float-bits (aref a 1)))
<aeth>
could maybe be optimized to produce similar asm to (defun bar (a) (declare (type (simple-array (unsigned-byte 32) (3)) a)) (aref a 1))
<aeth>
And if it isn't optimized, it still *works*
admich1 has quit [Ping timeout: 244 seconds]
<aeth>
A bit more useful of an optimization when it's e.g. joining two or more AREFs into one AREF of a different type if the multiple AREFs just merge the type (e.g. reading 4 octets from an octet array in a way that combines it into one (unsigned-byte 32) as if it was just an (unsigned-byte 32) array and one AREF)
admich1 has joined #commonlisp
<aeth>
I think I'm going to implement GPU structs this way: picking the most reasonable floating point or integer size, allocating a static-vector (probably as single-float or (unsigned-byte 32) unless it needs strange padding), and then just converting the types that don't fit in
veqq has quit [Quit: veqq]
<fe[nl]ix>
I don't know much about GPU computing, but it would be nice to have some MOP trickery to be able to define structs whole layout is compatible with FFI
<fe[nl]ix>
all unboxed
<fe[nl]ix>
are GPU structs different from C structs ?
chiselfuse has quit [Remote host closed the connection]
chiselfuse has joined #commonlisp
<aeth>
fe[nl]ix: you can somewhat control the layout if you generate your own SPIR-V as well (assuming all backends you're targeting have SPIR-V or something equally capable), but who knows what that does to the giant black box drivers and how much performance can pessimize if you stray from the default
<aeth>
It's actually much clearer that this can be done in GLSL than in SPIR-V's specification, but GLSL maps fairly directly to SPIR-V, so it has to be somewhere in there
anticomputer has quit [Remote host closed the connection]
anticomputer has joined #commonlisp
<aeth>
GLSL lets you choose between "packed", "shared", "std140", and "std430"
<aeth>
Searching "packed" in the huge SPIR-V spec that hurts some people's web browsers, it looks like you have the ability to do GLSLShared and GLSLPacked in Shader mode and CPacked in Kernel. https://registry.khronos.org/SPIR-V/specs/unified1/SPIRV.html
<ixelp>
SPIR-V Specification
lusciouslover has quit [Read error: Connection reset by peer]
lusciouslover has joined #commonlisp
nani has quit [Server closed connection]
nani has joined #commonlisp
<fe[nl]ix>
interesting
<aeth>
what's interesting is that the existence of "shared" implies that if something is optimized away (for being unused), then it may change the memory layout
shawnw has joined #commonlisp
<aeth>
It's "shared" as in, don't optimize it away so I can share this buffer even if it has the unused global variable
<aeth>
(share it between ones that have and don't have it)
<aeth>
If this is the case, you might have to ship an entire compiler with your program in order for the accessors to match the layout!
ingeniot has joined #commonlisp
istewart has quit [Quit: Konversation terminated!]
ingeniot has quit [Ping timeout: 248 seconds]
edgar-rft has joined #commonlisp
edgar-rft` has quit [Ping timeout: 272 seconds]
inline has joined #commonlisp
pve has joined #commonlisp
rgherdt has joined #commonlisp
bpanthi977 has quit [Ping timeout: 260 seconds]
bpanthi977 has joined #commonlisp
bpanthi977 has quit [Ping timeout: 276 seconds]
mgl has joined #commonlisp
fosskers has joined #commonlisp
fosskers has quit [Remote host closed the connection]
mgl has quit []
admich1 has quit [Ping timeout: 265 seconds]
ixelp has quit [Ping timeout: 252 seconds]
admich1 has joined #commonlisp
ixelp has joined #commonlisp
bpanthi977 has joined #commonlisp
bpanthi977 has quit [Ping timeout: 260 seconds]
treflip has joined #commonlisp
attila_lendvai has joined #commonlisp
bpanthi977 has joined #commonlisp
bpanthi977 has quit [Ping timeout: 276 seconds]
apac has joined #commonlisp
admich1 has quit [Read error: Connection reset by peer]
admich1 has joined #commonlisp
rolling-lambda_ has quit [Quit: Bye!]
treflip has quit [Ping timeout: 260 seconds]
rolling-lambda has joined #commonlisp
lcn_ has joined #commonlisp
<contrapunctus>
I was testing an expression like this - `(loop :for message :across messages :do (format t "~S ~S ~S~%" recipient title (gethash "to_display_name" message) (gethash "title" message) (~> (gethash "sent_on" message) (cl-date-time-parser:parse-date-time _) (lt:universal-to-timestamp _) (lt:format-rfc1123-timestring t _)))`
<contrapunctus>
And instead of getting `"message recipient" "message title" "Wed, 16 Jul 2025 03:31:01 +0530"`, I get `Wed, 16 Jul 2025 03:31:01 +0530"message recipient" "message title" "Wed, 16 Jul 2025 03:31:01 +0530"`
<contrapunctus>
Any idea why that's happening?
<jackdaniel>
I don't understand the syntax with ~> (some arrow macros?), it seems taht a few things are wrong
<jackdaniel>
first of all, your format specifies three ~s ~s ~s arguments, but you pass more parameters
<jackdaniel>
then, in format-rfc1123-... you use T, and I assume that it will write to standard output (consider NIL instead)
<jackdaniel>
also it is mighty hard to parse code pasted in irc window
pranav has quit [Remote host closed the connection]
random-nick has joined #commonlisp
<contrapunctus>
jackdaniel: Oh, my bad... ~> is a threading macro from Serapeum. `(~> (foo arg) (bar _) (quux _))` is the same as `(quux (bar (foo arg)))`.
jackdaniel has quit [Server closed connection]
<contrapunctus>
... _and_ I messed up the example I was sending, it was supposed to be `(format t "~S ~S ~S~%" (gethash "to_display_name" message) (gethash "title" message) (~> (gethash "sent_on" message) (cl-date-time-parser:parse-date-time _) (lt:universal-to-timestamp _) (lt:format-rfc1123-timestring t _)))` i.e. only three arguments
jackdaniel has joined #commonlisp
<contrapunctus>
And it was indeed the T passed to `format-rfc-1123`, thanks. Don't know why I didn't notice that.
cercopith has quit [Remote host closed the connection]
treflip has joined #commonlisp
apac has quit [Ping timeout: 245 seconds]
Lord_of_Life_ has joined #commonlisp
Lord_of_Life has quit [Ping timeout: 265 seconds]
Lord_of_Life_ is now known as Lord_of_Life
<pl>
anyone got an XL1200 or NXP1000 and could dump ROMs (and preferably a drive image) for me? :D
rkazak2 has joined #commonlisp
rkazak has quit [Ping timeout: 245 seconds]
X-Scale has joined #commonlisp
ixelp has quit [Ping timeout: 276 seconds]
ixelp has joined #commonlisp
bpanthi977 has joined #commonlisp
bpanthi977 has quit [Ping timeout: 276 seconds]
admich1 has quit [Remote host closed the connection]
admich1 has joined #commonlisp
notzmv has joined #commonlisp
X-Scale has quit [Ping timeout: 252 seconds]
attila_lendvai has quit [Ping timeout: 248 seconds]
beach` has joined #commonlisp
beach has quit [Killed (NickServ (GHOST command used by beach`!~user@2a01:cb19:682:4600:f0ea:cdea:529c:e0a5))]
beach` is now known as beach
TMA has quit [Ping timeout: 276 seconds]
skin has joined #commonlisp
TMA has joined #commonlisp
apac has joined #commonlisp
notzmv has quit [Ping timeout: 276 seconds]
treflip has quit [Ping timeout: 252 seconds]
notzmv has joined #commonlisp
dra has joined #commonlisp
dra has quit [Changing host]
dra has joined #commonlisp
decweb has joined #commonlisp
bpanthi977 has joined #commonlisp
tjmacs has joined #commonlisp
notzmv has quit [Ping timeout: 244 seconds]
tjmacs has quit [Remote host closed the connection]
treflip has joined #commonlisp
treflip has quit [Ping timeout: 252 seconds]
mwnaylor has quit [Ping timeout: 276 seconds]
rootnode has quit [Server closed connection]
rootnode has joined #commonlisp
rootnode has joined #commonlisp
rootnode has quit [Changing host]
Fade has quit [Changing host]
Fade has joined #commonlisp
notzmv has joined #commonlisp
inline has quit [Read error: Connection reset by peer]
chkhd` has quit [Quit: zzzZZZzzz]
jonatack has joined #commonlisp
treflip has joined #commonlisp
mwnaylor has joined #commonlisp
chkhd has joined #commonlisp
apac has quit [Ping timeout: 248 seconds]
inline has joined #commonlisp
mwnaylor has quit [Read error: Connection reset by peer]
notzmv has quit [Ping timeout: 248 seconds]
mwnaylor has joined #commonlisp
treflip has quit [Ping timeout: 276 seconds]
mesaoptimizer has quit [Server closed connection]
veqq has joined #commonlisp
mesaoptimizer has joined #commonlisp
admich1 has quit [Read error: Connection reset by peer]
admich1 has joined #commonlisp
calx-87 has quit [Ping timeout: 272 seconds]
cmack has quit [Remote host closed the connection]
domovod has joined #commonlisp
Lycurgus has joined #commonlisp
admich1 has quit [Ping timeout: 252 seconds]
admich1 has joined #commonlisp
domovod has quit [Quit: WeeChat 4.6.3]
Lycurgus has quit [Quit: irc.renjuan.org (juan@acm.org)]
calx-87 has joined #commonlisp
notzmv has joined #commonlisp
triffid has quit [Remote host closed the connection]
jonatack has quit [Ping timeout: 252 seconds]
RDMengineer has joined #commonlisp
gooba has quit [Ping timeout: 252 seconds]
triffid has joined #commonlisp
Thermoriax has quit [Remote host closed the connection]
gooba has joined #commonlisp
garlic0x1 has quit [Quit: Ping timeout (120 seconds)]