klange changed the topic of #osdev to: Operating System Development || Don't ask to ask---just ask! || For 3+ LoC, use a pastebin (for example https://gist.github.com/) || Stats + Old logs: http://osdev-logs.qzx.com New Logs: https://libera.irclog.whitequark.org/osdev || Visit https://wiki.osdev.org and https://forum.osdev.org || Books: https://wiki.osdev.org/Books
jcea has joined #osdev
itrsea has quit [Remote host closed the connection]
itrsea has joined #osdev
tigerbrother has quit [Quit: Ping timeout (120 seconds)]
tigerbrother has joined #osdev
guideX_ has joined #osdev
edr has quit [Quit: Leaving]
criswell has joined #osdev
gog has quit [Quit: byee]
<PublicWiFi> when a vfs is looking for a file it first checks the mount points and ect and finds the disk with the file + its filesystem
<PublicWiFi> then passses that request to the fs
<PublicWiFi> but how does the fs usually reference the actual disk driver? are the driver_ops stored in some data structure the filesystem can access?
<PublicWiFi> nvm this is my OS i can do what I want, I'm just gonna add the disk driver struct as a member to my mounted disk struct lol
<clever> PublicWiFi: this is a trap that ive seen in some places, one small kernel ive used, expects a fs to map to exactly one block device, so mounting an fs requires a block device name, and calls the fs mount function with a pointer to an already opened block dev
<clever> PublicWiFi: and that kernel has no way to iterate over FS's, so things like lvm or zfs searching for other members of a raid, just isnt possible
<clever> iterate over block devs*
<kof673> eh........couldn't you just make a pseudo block device that is really mapped to whatever raid scheme or concatentation etc. ? not saying that is cleaner lol
<kof673> *catenation
<kof673> i mean, freebsd could've gone "everything is a geom"
<clever> kof673: for mdadm sure, lvm yeah, but zfs no
<kof673> ^^^ :D
<kof673> zfs is special
<kof673> not saying that is bad, just it seems like "everything is a X" what is this x? :D
<clever> zfs is both raid and a fs, so it needs to open many block devices
criswell has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<Mutabah> Go for an object-oriented approach, and just have filesytem objects opaquely hold a handle to whatever backs them (and maybe a handle to a generic filesystem cache)
<clever> for zfs, you basically have a giant tree of metadata, each pointing to more children, and all metadata is immutable
<clever> any time you want to modify some metadata, you create the new version in what was free space, and recursively run up the tree and rewrite it as well
<clever> the only metadata at a fixed position is a ring of pointers to the root of the tree, each root is a different version
criswell has joined #osdev
<clever> in raidz mode, a record gets split into chunks, parity added, and then the chunks+parity gets spread over several disks
<clever> the other mode (i forget the name), basically just throws the data onto a random "vdev"
<clever> where a vdev can be either a raidz1 set of disks, or a non-redundant disk
<PublicWiFi> hmm
<PublicWiFi> ok so in a more "perfect world" whats the flow
<PublicWiFi> user touches a new file
<PublicWiFi> how does the filesystem actually talk to the disk? What's the usually plan of attack of this abstraction?
<PublicWiFi> i mean ofc I have a driver for the disk already done
<clever> PublicWiFi: what ive seen, is that the block device has functions to read/write a range of blocks at a given block# within the blk-dev
<clever> a partition table driver would parse the MBR/GPT tables, and then create more blkdevs that just apply an offset to all transfers, and check the pos against a size
<clever> LVM is fairly simple too, its just a table of mappings, range X-Y of blk dev foo/bar maps to range A-B of physical disk C, just a big list of those
<PublicWiFi> right so your blockdev entry will have your dev_ops or whatever
<clever> yep
<clever> and then a physical disk driver, would do the actual IO to the hw
<PublicWiFi> sorry I keep meaning to forget that im doing a really cut down OS for an 8-bit device lol
<PublicWiFi> right
<PublicWiFi> so where Im lost is how the filesystem actually calls the device driver, would it just reference the devblk i guess?
<clever> yeah
<bslsk05> ​github.com: lk-overlay/platform/bcm28xx/sdhost/sdhost_impl.cpp at master · librerpi/lk-overlay · GitHub
<clever> here is my SD card driver for the rpi
<bslsk05> ​github.com: lk-overlay/platform/bcm28xx/sdhost/block_device.hpp at master · librerpi/lk-overlay · GitHub
<PublicWiFi> ah ok so you're passing the bdev entry to the read block
<clever> in this case, i subclass the bdev_t from the kernel i'm using
<clever> so you can cast my SD management class into a bdev_t, and then it only reads the first half of the member variables
<PublicWiFi> ah gotcha ok
<clever> so it has the read/write methods the base kernel is expecting of every block dev
<PublicWiFi> okay so what do you do to keep up with your mount points?
<clever> the kernel handles that, one min
<bslsk05> ​github.com: lk/lib/fs/fs.c at master · littlekernel/lk · GitHub
<PublicWiFi> okok perfect
<PublicWiFi> has the path, the bdev entry, and etc
<PublicWiFi> okok this is starting to click now
<clever> PublicWiFi: LK keeps a linked list of `struct fs_mount`, which has a pointer to the fs implementation, a cookie (the fs implementation makes one for each mount), and yeah, the bdev and others
<clever> fs.c then walks that linked list, to know which fs to talk to
<nikolar> u
<bslsk05> ​github.com: lk/lib/fs/ext2/ext2.c at master · littlekernel/lk · GitHub
<clever> PublicWiFi: when LK mounts an fs, it calls the mount function, and passes it an already open bdev_t
<PublicWiFi> ahh ok gotcha!
<PublicWiFi> this is helping, thank you :)
<clever> and the mount function must return a cookie, and that is then passed to all over functions
<clever> so the ext2 driver stores the state on a given mount, within that cookie
<PublicWiFi> ah ok I see!
<clever> s/over/other/
<bslsk05> ​<clever*> and the mount function must return a cookie, and that is then passed to all other functions
<clever> PublicWiFi: so you can see on line 106 of ext2.c, it uses bio_read to read the bdev_t at byte offset 1024 i think
<PublicWiFi> yeah I think I understand!
<bslsk05> ​github.com: lk/lib/bio/bio.c at master · littlekernel/lk · GitHub
<clever> and bio_read is just a wrapper, that calls the ->read function on the bdev_t
<bslsk05> ​github.com: lk/lib/fs/ext2/ext2.c at master · littlekernel/lk · GitHub
<clever> PublicWiFi: and over here, all of the entry-points for ext2 are defined
<bslsk05> ​github.com: lk/lib/fs/ext2/file.c at master · littlekernel/lk · GitHub
<clever> PublicWiFi: the ext2_lookup here, then walks the directories of ext2, as it also looks thru the given path, until it knows the inode that is being opened
<clever> for ext2/3/4, the inode table is a series of arrays, each array being `s_inodes_per_group` large, and then something something each array starts at some offset?
<PublicWiFi> unforunately my inode cache is gonna be like
<PublicWiFi> 8 large
<PublicWiFi> :D
<clever> for little-kernel...
<bslsk05> ​github.com: lk/lib/fs/ext2/ext2.c at master · littlekernel/lk · GitHub
jcea has quit [Ping timeout: 252 seconds]
<PublicWiFi> ext2 might still end up being too much
<clever> it creates a block cache, which is basically just a bdev_t proxy
<PublicWiFi> right now im runnin fat16 but its not great
<clever> any recently read block stays in that cache
<clever> and it evicts things as misses pull other blocks in
<PublicWiFi> and just putting a small compat layer inbetween to treat it like ithas inodes
<clever> for zfs, the inode table is functionally just a file
<PublicWiFi> interesting
<clever> so any reads of that table, go thru the same mechanics as reading any other file
<PublicWiFi> zfs is basically black magic to me at this point, even as a sysadmin
<PublicWiFi> i haven't gotten around to actually learning how it works
<clever> the MOS (meta object set) has a pointer to the root inode for each filesystem
<clever> which you can then read like any other file
skipwich has quit [Remote host closed the connection]
skipwich has joined #osdev
itrsea has quit [Remote host closed the connection]
itrsea has joined #osdev
criswell has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
TkTech0 has joined #osdev
TkTech has quit [Ping timeout: 252 seconds]
TkTech0 is now known as TkTech
frkzoid has quit [Read error: Connection reset by peer]
MarchHare has quit [Quit: Leaving]
frkazoid333 has joined #osdev
GeDaMo has joined #osdev
particleflux has quit [Server closed connection]
particleflux has joined #osdev
sidcha64 has quit [Quit: The Lounge - https://thelounge.chat]
sidcha649 has joined #osdev
Jari-- has joined #osdev
<Jari--> hi all dudes
itrsea has quit [Remote host closed the connection]
itrsea has joined #osdev
goliath has joined #osdev
Jari-- has quit [Ping timeout: 255 seconds]
netbsduser` has quit [Ping timeout: 260 seconds]
TkTech2 has joined #osdev
TkTech has quit [Ping timeout: 252 seconds]
TkTech2 is now known as TkTech
Lucretia has joined #osdev
mavhq has quit [Ping timeout: 248 seconds]
simpl_e has quit [Server closed connection]
simpl_e has joined #osdev
mavhq has joined #osdev
<nikolar> clever: where did you learn all those details about zfs internals
<geist> oddly enough i asked gemini some pretty technical details about some zfs stuff (layout of the vdev cache device, etc) and it was pretty good at giving me info
<geist> helps if you know precisely what you're asking for, but if you probe it like that it can give you good stuff
Left_Turn has joined #osdev
f1sty has joined #osdev
TkTech1 has joined #osdev
TkTech has quit [Ping timeout: 248 seconds]
TkTech1 is now known as TkTech
gog has joined #osdev
gog has quit [Client Quit]
karenw has joined #osdev
guideX_ is now known as guideX
karenw has quit [Remote host closed the connection]
karenw has joined #osdev
itrsea has quit [Remote host closed the connection]
itrsea has joined #osdev
EmanueleDavalli has joined #osdev
Lucretia has quit [Remote host closed the connection]
criswell has joined #osdev
Celelibi has quit [Ping timeout: 240 seconds]
karenw_ has joined #osdev
karenw has quit [Ping timeout: 245 seconds]
karenw_ has quit [Ping timeout: 272 seconds]
karenw_ has joined #osdev
karenw_ has quit [Client Quit]
f1sty has quit [Quit: leaving]
f1sty has joined #osdev
navi has quit [Server closed connection]
navi has joined #osdev
Celelibi has joined #osdev
criswell has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Matt|home has quit [Ping timeout: 260 seconds]
criswell has joined #osdev
Lucretia has joined #osdev
criswell has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
criswell has joined #osdev
edr has joined #osdev
f1sty has quit [Quit: leaving]
itrsea has quit [Remote host closed the connection]
itrsea has joined #osdev
Maja has quit [Server closed connection]
Maja has joined #osdev
bauen1 has quit [Ping timeout: 255 seconds]
nur has quit [Read error: Connection reset by peer]
iminyourwalls_ is now known as iminyourwalls
bauen1 has joined #osdev
nur has joined #osdev
nur has quit [Remote host closed the connection]
nur has joined #osdev
m3a has quit [Server closed connection]
m3a has joined #osdev
EmanueleDavalli has quit [Ping timeout: 260 seconds]
goliath has quit [Quit: SIGSEGV]
msv has quit [Ping timeout: 260 seconds]
EmanueleDavalli has joined #osdev
itrsea has quit [Remote host closed the connection]
itrsea has joined #osdev
msv has joined #osdev
Turn_Left has joined #osdev
Left_Turn has quit [Ping timeout: 255 seconds]
jimbzy has joined #osdev
svm has joined #osdev
netbsduser` has joined #osdev
msv has quit [Ping timeout: 260 seconds]
svm is now known as msv
sprock has joined #osdev
sprock has quit [Ping timeout: 255 seconds]
sprock has joined #osdev
ThinkT510 has quit [Ping timeout: 276 seconds]
ThinkT510 has joined #osdev
ThinkT510 has quit [Ping timeout: 248 seconds]
da5id has joined #osdev
itrsea has quit [Read error: Connection reset by peer]
itrsea has joined #osdev
ThinkT510 has joined #osdev
ThinkT510 has quit [Read error: Connection reset by peer]
xenos1984 has quit [Ping timeout: 252 seconds]
ThinkT510 has joined #osdev
<clever> nikolar: mostly i found a pdf about the on-disk format
<nikolar> ah fun
<clever> nikolar: just google for Zfs_ondiskformat.pdf and you should find it
<nikolar> cool, thanks :)
aaabbb has quit [Server closed connection]
aaabbb has joined #osdev
xenos1984 has joined #osdev
bauen1 has quit [Ping timeout: 252 seconds]
xenos1984 has quit [Ping timeout: 252 seconds]
netbsduser` has quit [Ping timeout: 260 seconds]
liney has quit [Server closed connection]
liney has joined #osdev
xenos1984 has joined #osdev
chibill has quit [Read error: Connection reset by peer]
chibill has joined #osdev
EmanueleDavalli has quit [Quit: Leaving]
bauen1 has joined #osdev
alpha2023 has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
alpha2023 has joined #osdev
Turn_Left has quit [Read error: Connection reset by peer]
gildasio has quit [Ping timeout: 240 seconds]
Turn_Left has joined #osdev
gildasio has joined #osdev
Turn_Left has quit [Read error: Connection reset by peer]
mx08 has quit [Quit: WeeChat 3.8]
mx08 has joined #osdev
Turn_Left has joined #osdev
SophiaNya has quit [Server closed connection]
SophiaNya has joined #osdev
GeDaMo has quit [Quit: 0wt 0f v0w3ls.]
ptrc has quit [Server closed connection]
ptrc has joined #osdev
itrsea has quit [Remote host closed the connection]
itrsea has joined #osdev
karenw_ has joined #osdev
Yoofie6464463823 has quit [Ping timeout: 252 seconds]
gf31 has joined #osdev
gf3 has quit [Read error: Connection reset by peer]
gf31 is now known as gf3
randm has quit [Remote host closed the connection]
randm has joined #osdev
agent314 has joined #osdev
gf3 has quit [Ping timeout: 245 seconds]
bauen1 has quit [Ping timeout: 255 seconds]
bauen1 has joined #osdev
criswell has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
criswell has joined #osdev
criswell has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Lucretia has quit [Remote host closed the connection]
fgarcia_ has joined #osdev
fgarcia has quit [Ping timeout: 252 seconds]
agent314 has quit [Quit: WeeChat 4.5.2]
criswell has joined #osdev