<kchanqvq>
When list-alloc-tramp shows up in the majority of a function's sb-sprof vertices, does that mean it's spending most of its time CONSing lists?
<kchanqvq>
I'm consing lots of fairly small (2~3 elements) lists and I didn't expect to find it as bottleneck
jonatack has joined #commonlisp
random-nick has joined #commonlisp
<fosskers>
Consider `values` if you can. Yes little conses do add up.
<fosskers>
I was able to drastically reduce allocation with parcom when I made that change.
jon_atack has joined #commonlisp
<kchanqvq>
It's used for hash-consing. I'm currently always CONSing them up to pass to test in an EQUAL hashtable, and they'll be retained if not already present, so I guess I can't use VALUES. But I guess maybe I can try allocate on the stack first, and copy it out only when I need to retain?
jonatack has quit [Ping timeout: 260 seconds]
<kchanqvq>
Is list-alloc-tramp always called when CONSing or is it on some slow path?
<fosskers>
As far as I saw when I was doing a lot of benchmarking, it was that.
triffid has joined #commonlisp
<fosskers>
Yeah you could try `dynamic-extent`.
Lord_of_Life has quit [Ping timeout: 252 seconds]
Lord_of_Life has joined #commonlisp
mgl has joined #commonlisp
jonatack has joined #commonlisp
jon_atack has quit [Ping timeout: 260 seconds]
jon_atack has joined #commonlisp
jonatack has quit [Ping timeout: 276 seconds]
X-Scale has quit [Ping timeout: 260 seconds]
yitzi has joined #commonlisp
jonatack has joined #commonlisp
jon_atack has quit [Ping timeout: 240 seconds]
King_julian has quit [Ping timeout: 252 seconds]
King_julian has joined #commonlisp
decweb has joined #commonlisp
gooba has quit [Remote host closed the connection]
gooba has joined #commonlisp
jon_atack has joined #commonlisp
jonatack has quit [Ping timeout: 272 seconds]
fosskers has quit [Remote host closed the connection]
jonatack has joined #commonlisp
gooba has quit [Remote host closed the connection]
gooba has joined #commonlisp
jon_atack has quit [Ping timeout: 245 seconds]
jon_atack has joined #commonlisp
King_julian has quit [Ping timeout: 276 seconds]
jonatack has quit [Ping timeout: 248 seconds]
lucerne has quit [Quit: Bye]
lucerne has joined #commonlisp
X-Scale has joined #commonlisp
jonatack has joined #commonlisp
jon_atack has quit [Ping timeout: 245 seconds]
cage has joined #commonlisp
jon_atack has joined #commonlisp
jonatack has quit [Ping timeout: 252 seconds]
robin has quit [Ping timeout: 248 seconds]
Lycurgus has joined #commonlisp
robin has joined #commonlisp
prokhor has quit [Remote host closed the connection]
stanrifkin has joined #commonlisp
treflip has quit [Ping timeout: 244 seconds]
treflip has joined #commonlisp
RavenJoad has joined #commonlisp
<RavenJoad>
I am checking the argument to a function with check-type. I want the variable to either be a string or a pathname. I also want to allow a function that produces one of those too. What is the best way to write this type check?
<beach>
I don't think you can do better than (or string pathname function)
<beach>
"The list form of the FUNCTION type-specifier can be used only for declaration and not for discrimination"
<bike>
in other words, there's no way to determine that a function always returns an object of a given nontrivial type. because of how lisp types work, doing so is technically mathematically impossible
<beach>
"undecidable"
<beach>
RavenJoad: Does that make sense?
<decweb>
Could he perhaps do something with `satisfies` here>
<decweb>
s/>/?/
<RavenJoad>
Yeah, it does. Tis the price we pay for runtime dynamism like this. It's not important.
rgherdt has quit [Read error: Connection reset by peer]
<beach>
decweb: How would that work?
rgherdt has joined #commonlisp
<beach>
RavenJoad: "price we pay"? Rather the advantage of not having to supply type information all the time.
admich1 has quit [Remote host closed the connection]
admich1 has joined #commonlisp
<RavenJoad>
Fair enough, it cuts both ways. I'm an idiot, so I like having the guard-rail of clearly knowing the type while iterating.
<beach>
RavenJoad: Is any string and any pathname acceptable?
puke has quit [Quit: puke]
<RavenJoad>
More or less. The particular thing I was working on is a with-named-temporary-file macro (which behaves similar to uiop:with-temporary-file) so I can create temporary files and optionally chose to remove them. The validation is currently happening in a with-* macro (but should probably move to the call-with-* function.
<beach>
I am asking because PATHNAME seems a bit too permissive. Like can it have any host, any device, any version, etc?
<decweb>
Beach, I was mostly asking, I rarely use satisfies and whatever limitations are inherent in the combination of check-type and satisifes as "type specifiers" were not obvious to me from the hyperspec.
<beach>
decweb: I see.
Lycurgus has quit [Quit: irc.renjuan.org (juan@acm.org)]
bpanthi977 has quit [Ping timeout: 260 seconds]
jonatack has joined #commonlisp
jon_atack has quit [Read error: Connection reset by peer]
Spawns_Carpeting has quit [Quit: ZNC 1.8.2+deb3.1+deb12u1 - https://znc.in]
<RavenJoad>
PATHNAME is definitely too permissive. But it's enough to remind me what I am supposed to pass. This is a utility for running unit tests, so technically all paths would be completely local.
<beach>
RavenJoad: Then you are not taking advantage of the excellent Common Lisp type system, and you are restricting yourself to types that could be expressed in a statically typed language.
<beach>
Anyway, I'll quit now. Sorry for harassing you about this.
<RavenJoad>
I am definitely not taking advantage of everything in the type system, that is for sure. I don't know what more I want. I just whipped this macro together to make my difference unit tests work.
lusciouslover has quit [Quit: \]
treflip has quit [Remote host closed the connection]
<mwnaylor>
Nevermind, figured it out. I had to provide the fully qualified symbol for the slot. In this case, it was 'stumpwm::split-ratio.
wohonajax has joined #commonlisp
akoana has joined #commonlisp
<wohonajax>
i've been doing a little testing on methods for collecting a string. VECTOR-PUSH on a string with a fill pointer is vastly more efficient than anything else, including FORMAT with a string with a fill-pointer as the stream argument and (especially) WITH-OUTPUT-TO-STRING
<acdw>
why not rewirte w/output-to-string to use vector-push
<wohonajax>
that's exactly what i ended up doing in my code, unless you mean an implementation should do that
<acdw>
oh nice! i mean ideally yues, but if you're like me you're not on a impl's team
<wohonajax>
i suppose the intermediary stream creation introduces overhead somehow or maybe the underlying implementation of streams could be made more efficient
decweb has quit [Remote host closed the connection]
<wohonajax>
i did peek at sbcl's source code on github but it's a bit arcane and low-level for me to read...
admich1 has quit [Read error: Connection reset by peer]
chiselfuse has quit [Remote host closed the connection]
<wohonajax>
i should also note that all this testing was in fact done only on sbcl so i have no idea about other implementations still
admich1 has joined #commonlisp
chiselfuse has joined #commonlisp
szkl has joined #commonlisp
triffid has quit [Remote host closed the connection]
rkazak has joined #commonlisp
rkazak has quit [Client Quit]
ndudaev has left #commonlisp [WeeChat 3.8]
triffid has joined #commonlisp
jon_atack has joined #commonlisp
jonatack has quit [Ping timeout: 252 seconds]
admich1 has quit [Read error: Connection reset by peer]
admich1 has joined #commonlisp
veqq has quit [Quit: veqq]
veqq has joined #commonlisp
jonatack has joined #commonlisp
jon_atack has quit [Ping timeout: 252 seconds]
decweb has joined #commonlisp
<aeth>
maybe because streams are extensible and because it doesn't know what kind of stream it is when in FORMAT since WITH-OUTPUT-TO-STRING is separate from FORMAT
<aeth>
while VECTOR-PUSH within the same function that the vector is created might be able to know everything about the vector (including size! you said -PUSH not the more likely -PUSH-EXTEND, which the stream might be using)
<aeth>
FORMAT with a string with a fill pointer could and maybe should be identical to the VECTOR-PUSH code, though. Especially if the string is created in the same function that the FORMAT is in
lusciouslover has quit [Quit: \]
<wohonajax>
i did test VECTOR-PUSH-EXTEND too and while it was slightly slower than VECTOR-PUSH it was still way faster than WITH-OUTPUT-TO-STRING
lusciouslover has joined #commonlisp
<aeth>
odd, then, because VECTOR-PUSH-EXTEND should be entirely safe
<aeth>
like, you could just rewrite the other two in V-P-E (which is what I tend to use to build strings) unless there are edge cases that they're slowed down to consider
<aeth>
wohonajax: oh wait, there's one more caveat, you'd have to do a copy at the end, right? to get a simple-string
<aeth>
Because COPY-SEQ is the portable way to get a simple-array from one that might not be. Implementations should be able to beat this because they might be able to cheat and make adjustable arrays that become no longer adjustable once returned to the user.
<wohonajax>
i was testing everything with (make-array 0 :element-type 'character :fill-pointer 0)
bpanthi977 has quit [Ping timeout: 276 seconds]
<aeth>
Yes, which isn't a simple-string, so if an implementation is writing generic code, they'll probably do COPY-SEQ (or an internal hack) at the end to give the end user the most optimized end type. I don't think it's required, though.
<aeth>
,(typep (with-output-to-string (s) (format s "Hello")) 'simple-string)
<ixelp>
=> T
<aeth>
So that's both CCL and SBCL that make those simple-strings
kchanqvq has joined #commonlisp
<aeth>
wohonajax: So it's not quite apples-to-apples unless you COPY-SEQ at the end (which creates a non-adjustable string of the final size instead of leaking the implementation of the string-building) if your algorithm doesn't produce simple-strings.
<aeth>
(with no fill-pointer, too)
<kchanqvq>
Is there anything I can do to speedup EQUAL hash table if I only put in list of structs (for hash-consing)?
<kchanqvq>
Benchmark is showing I spent most time on EQUAL hash table. This ought to do better!
<wohonajax>
aeth: after testing it looks like COPY-SEQ didn't make an appreciable difference for the V-P code
<aeth>
Hmm... Technically speaking, I think an optimization should be able to be present for the pattern of a COPY-SEQ on an adjustable and fill-pointer array to make it into a simple-array, which is that if it knows no other references to it exists (if it's done within one function), it can probably just reuse the old array during the COPY-SEQ, effectively dealloacting and reallocating with a different array
<aeth>
type prefix (although it'd have garbage after the end of the fill-pointer to somehow have to know to collect).
<aeth>
Especially if done within an internal, unsafe, low-level function. This would make rewriting the other string-builder functions into VECTOR-PUSH-EXTEND with a COPY-SEQ at the end into a no-brainer.
<kchanqvq>
scymtym: I think I already have adaptive hashing. The problem with custom hash function seem to be require they don't change, so I can't use SB-IMPL::EQ-HASH... because they change after GC, and custom hash-table don't provide customization for rehashing behavior? I'll look at the link closer
<wohonajax>
this reminds me of push-nreverse optimization, google AI told me that sbcl does the optimization and i think i read in a book (maybe PCL) that smart compilers will optimize it into something like a loop collect. i tested it some and it does seem to get optimized in some cases though i couldn't find (and probably couldn't understand) the source code for the optimization
<acdw>
how does loop collect optimize it? i assumed it was push nreverse under the hood lol
<acdw>
i guess like. what even is the alternative to doiong that
<bike>
no, loop collect does something completely different, it keeps a pointer to the end of the list and keeps adding new conses on the end
<wohonajax>
my understanding is that it keeps a pointer to the final CDR and REPLACD/SETFs it
<bike>
yes.
<wohonajax>
avoiding traversing the list again
<bike>
i don't know what you mean by "push-nreverse optimization". If you mean converting a series of pushes followed by an nreverse into something loop collect, i'm pretty sure it does not do that.
<bike>
"something like"
jon_atack has joined #commonlisp
<kchanqvq>
scymtym: interesting, I happen to be writing an egraph engine as well! Hope I can discuss with hayley some time :D
<aeth>
bike: yeah, that's how I'd assume it would work
jonatack has quit [Ping timeout: 272 seconds]
<aeth>
and it's a lot easier to use collect than to track CONSes yourself
<aeth>
Doing REVERSE is more of a Scheme thing because they prefer FP, recursion, and not mutating... and they may even have immutable cons pairs by default with a separate mcons/mlist available. But that wouldn't be NREVERSE either
<scymtym>
kchanqvq: i haven't thought aout this in detail but for hash-consing i would assume that equality and hash code based on eq or anything that would require integration with the gc, really, defeats the purpose
<aeth>
Of course, with immutable lists, your implementation could still treat them as mutable while building them, which would give LOOP even more of an advantage over doing it yourself
<aeth>
(Which is merely hypothetical, of course, because I doubt Common Lisp is adding icons/ilist anytime soon. Although nothing's stopping them from doing it and exposing it to a trivial-foo library)
bpanthi977 has joined #commonlisp
jonatack has joined #commonlisp
dra has joined #commonlisp
dra has quit [Changing host]
dra has joined #commonlisp
<kchanqvq>
scymtym: really, why do you think it defeats the purpose? The purpose is fast :) Thanks for the link btw I see the trick is just to use SXHASH! A few years ago SXHASH in SBCL returns constant for all struct but now I tested it's much better. I looked at code and see %INSTANCE-SXHASH integrates with GC to ensure stable hash code. I guess this
<kchanqvq>
applies to struct?
jon_atack has quit [Ping timeout: 244 seconds]
<scymtym>
kchanqvq: i assumed the goal was to identify structurally identical but distinct objects. maybe i misunderstood what you are doing
<kchanqvq>
scymtym: we ensure hash cons invariant recursively, so element of the list (term) are already hash-consed, thus EQ is enough and no recursive descending needed
<scymtym>
kchanqvq: i see. makes sense
<kchanqvq>
scymtym: then there's a convenient function to recursively hash-cons a non-recursively-hash-consed structure user gives. But the terms stored in the hash-cons has this invariant, which is the faster and more memory efficient way of doing hash-cons
<scymtym>
couldn't you spent a little memory to store an arbitrary, unchanging hash, possibly initially identical to eq-hash, in each canonical instance and use that when computing compound hashes?
<kchanqvq>
scymtym: Yes, that's possible as well and I have tried it. I'm just trying to squeeze the most performance. I'm going to compare these approaches a bit!
Catie has quit [Read error: Connection reset by peer]
Catie` has joined #commonlisp
kchanqvq64 has joined #commonlisp
Lycurgus has joined #commonlisp
kchanqvq has quit [Ping timeout: 252 seconds]
admich1 has quit [Ping timeout: 252 seconds]
admich1 has joined #commonlisp
<kchanqvq64>
scymtym: surprisingly using (incf *hash-code*) makes things much slower. Maybe this is just a very bad hash
<kchanqvq64>
Oh using sb-c::mix instead of logxor brings speed back to normal. sb-c::mix is a very good mix function
<kchanqvq64>
Turns out maintaining hash-code ourselves is about the same performance as SXHASH. But I think maintaining our own hash-code has more portable performance (some implementations still return constant for all structs).
jeffrey has quit [Ping timeout: 276 seconds]
jon_atack has joined #commonlisp
Spawns_Carpeting has quit [Quit: WeeChat 4.5.2]
Spawns is now known as Spawns_Carpeting
jonatack has quit [Ping timeout: 265 seconds]
kchanqvq has joined #commonlisp
kchanqvq64 has quit [Ping timeout: 252 seconds]
dra has quit [Quit: Leaving]
Lycurgus has quit [Quit: irc.renjuan.org (juan@acm.org)]
chiselfuse has quit [Remote host closed the connection]
chiselfuse has joined #commonlisp
chiselfuse has quit [Remote host closed the connection]
chiselfuse has joined #commonlisp
kchanqvq has quit [Ping timeout: 252 seconds]
admich1 has quit [Ping timeout: 260 seconds]
admich1 has joined #commonlisp
apac has joined #commonlisp
lcn_ has quit [Remote host closed the connection]
pve has quit [Quit: leaving]
Ruby has joined #commonlisp
jonatack has joined #commonlisp
jon_atack has quit [Ping timeout: 252 seconds]
apac has quit [Ping timeout: 248 seconds]
rtypo has quit [Ping timeout: 248 seconds]
kchanqvq has joined #commonlisp
<kchanqvq>
is 64 still the only good width for modular arithmetic nowadays? or is (logand ... most-positive-fixnum) good?
<aeth>
I only ever work in unsigned-byte 32 and 64, personally. In theory, fixnum stuff is more optimized, but in theory, implementations can also optimize some subset of ub64 even if it may be a bignum. Anything I wrote that mentions "fixnum" is probably 5+ years out of date from my current style.
<aeth>
I kind of wish implementations all agreed to also optimize some in-between like, say, 48. Almost certainly a fixnum, but also a known, portable size. But ub48 is probably going to round up to nonnegative-fixnum in arrays and then any operation that accesses that array is going to not know it's a ub48.
<kchanqvq>
aeth: I'm working on non-negative-fixnum in a hash function (SBCL's :hash-function require non-negative-fixnum). I hope this would just work. If not I can work on ub64 first then truncate at the end
<aeth>
oh, implementation-specific stuff
<kchanqvq>
oops, I forgot to mention SBCL at the start :P
<aeth>
well, I still reiterate that I wish they'd optimize ub48 as a special case even if you lose 14 bits over nonnegative-fixnums.
<aeth>
as an alternative to ub62 (in 64-bit x86-64 SBCL) or ub64