Wanda[cis] has quit [Quit: Idle timeout reached: 172800s]
Wanda[cis] has joined #prjcombine
<Wanda[cis]>
okay, I have migrated serialization off serde and fixed some other problems with it along the way
<Wanda[cis]>
the one problem with this is that it resulted in invalidating all caches I had, so I'm still waiting for all the databases to recompute from nil
<Wanda[cis]>
also, after having to fix up some fallout that resulted in broken bsdata databases several times, I have started thinking how to integrate caching into prjcombine-hammer in a way that wouldn't require excessive invalidations, particularly when something screws up the fragile prerequisites of the algorithm
<mupuf>
yeah, migration of DBs will become more important as time goes by, because replicating the extraction may get harder and harder
<Wanda[cis]>
yeahh, I need to actually come up with some satisfying way to make the extraction reproducible in the first place
<Wanda[cis]>
right now it's not even possible to use all the crap in re/ subdirectory outside of my own machine, or at least with a copy of ISE / icecube with the same patches applied
<Wanda[cis]>
of course, this is a bit of a problem given that you need to obtain a copy of the vendor tools in the first place, and some of them are ... hard to come by
<Wanda[cis]>
I have absolutely no recollection of how I obtained the copy of ISE 2.1 I'm using to reverse XC5200, for one
galibert[m] has joined #prjcombine
<galibert[m]>
may I suggest generating the data as a human readable text file, and building databases from that?
DemiMarieObenou4 has joined #prjcombine
<DemiMarieObenou4>
I like that approach
<Wanda[cis]>
this is not a viable strategy
<Wanda[cis]>
prjcombine extracts a ridiculous amount of raw data from the toolchains to do its thing
<galibert[m]>
Ah, that’s for a format in the middle, not the final results
<Wanda[cis]>
this is with binary serialization and zstd compression
<Wanda[cis]>
I also don't particularly enjoy writing text serialization and deserialization code when I can just #derive something that will roundtrip
<galibert[m]>
Sure
<Wanda[cis]>
it would be nice if someone invented a standard text serialization format that I can just serialize and deserialize without major pain
<Wanda[cis]>
I don't really like commiting a zstd'd blob as the main source of truth into the repo, but it seems to be the easiest way to get the job done
<Wanda[cis]>
I also emit the final database as a json file in parallel, which is a massive pain because json is not a good serialization format; I originally did it so that the python doc generator can read it, but it was dogshit so it got rewritten in mdbook+rust; nowadays I keep it only so git diff works