51-57:51- Starting logfile IrcLog.#wgs [ This log was trimmed of some extra unneeded info ] IRC log started Mon Feb 25 08:20:39 2002 jolz: explain JOS' current implementation of block devices and filesystems ok.. 51-57:51- Topic 50(36#wgs5650): changed by 36jbevren5650: JOS: block device; filesystem; rewrite log on block devices simply respond to ipc such as READB256, WRITEB256 which specify the logical block number and the memory to read it into i wanna change this though so they can just act like normal files and respond to READ/WRITE and SEEK as well that sums it uP? ah, that sums block devices up really but the block device SHOULD be nothing but read block, write block add query device of course that's elegant enough, though in its native form, UHS will use 512 byte blocks theres extra stuff for iec devies.. query and turn light on/off.. ok that sounds like a good idea btw when buffers arent clean for a device, have its LED on or flashing (that way someone doesnt yank a disk that's not clean) anyway jbevren: yeah thats how it is ok! also has a call to check if the media was swapped the filesystems only ever check this when the light would have been off now, ok thats basically it for the block device.. lets work from a base on how the system is built: you have multiple buses (especially inthe case of the ide64 systems) below, you have block devices on greg's sx, for example, you have an ide bus, and an iec bus (and a ramlink bus) yeah.. so, organize drivers such: bus-device-extraparms the extra parms are optional (used for devices that have partitions for example) there's drivers as such: an idebus.drv, and idedsk.drv (and idecd.drv) idebus talks to idedsk and idecd :) iecbus.drv, (1541.drv, 1571.drv, etc) see? except thats too much ipc.. this way we can have an arbiter for buses whyso? for the most part, the communication will be passing of header paramaters. the block of data doesnt have to move more than once because then its app->fsys + cache->device->bus->actual device but do answer :) ok, quesntion then, :) How do you manage multiple IEC drivers on the iec bus? they all follow the 1 protocol how do you keep one driver from using the port while another is? does it just block? theres only 1 driver iec.drv is it one driver for the 1541, 1571, 1581, fd2k, fd4k, blahdisk? ok yep is it a huge driver? :) jbevren: nope small driver Ok :)) I ask about the blocking, cause UHS will have as many as 7 devices, and not blockign the harddisks while waiting for tapes is, um, good :) the booter contains the drive code for every device and from that point on theyve all gotta follow the same protocol I see. hrm, ok. we'll stick with your model ;) but JOS needs to be made aware that not all devices are 256-byte sectored :) (cause I dont wanna deblock inside UHS's driver) =) jbevren: ide.dev uses 512 byte sectors cool:) READB512,WRITEB512 but i dont wanna have to make new codes for each sector size which is why i wanna change that right so, when a driver initializes (ide.drv for ex please), how does it inform the system of new block devices? assumption: there are drivers for cdrom and harddisk, and one of each is on the ide it simple adds a /dev/ entry for that device how does the /dev entry attach to the driver? is it a special file with a pointer to the driver? the driver makes it, its not a real file, only a virtual file right what I'm asking: Are there 'reserved' names? or does it go by some kernel-supplied ID? ok that's all I needed to know. jolz: can I make a subdir in /dev? :) Nate\DAC: btw, this is what linux users call 'devfs') jbevren: no reserved names jbevren: sure if you wanted to cool ok, so what happenes between fopen("/dev/ide0"); and the ide block driver getting notified? an IO_OPEN message is sent to the driver, with the handle of the name, plus any extra path info, (for filesystems) jolz: perfect. I understand now jolz: is there an ioctl() call? jbevren: yep ioctl("/dev/ide0",O_EJECT) cool! then you're set. explain why, at this point, you have to have separate calls for readb256 and readb512 jbevren: no reason in particular, simplicity ok. heh so reading and writing data to the drive, at a specific block? :) does JOS use LBA or CHS?> -> lba thought so question is: How does the filesystem contact the device driver, and how does it specifiy a specific sector good. Uhs will be 100% lba if I can do anything about it :) how does readblock() work? first all a filesystem gets mount("/sys/filesystem", "/dev/device", "/mountpoint"); right 51-57:51- Topic 50(36#wgs5650): changed by 36jbevren5650: JOS: Filesystem<->block device comms after it gets the mount call, it opens the device specified, and might need to read a block depending on the filesystem translater from T/S or C/H/S is upto the filesystem k wait. t/s to c/h/s? that's the same :) to read a block, the filesystem goes through the filesystem libraries cache rountines to get it OH nevermind, forgot abot commodore's methods :) he said 'or' not 'to' ok :) the cache is a simple uses LRU to decide which blocks to overwrite or flush and has a hash table to quickly find the block and the block call gets to the driver how? brb all I understand so far is reading /dev/ide0 as a char device [ nate leaves ] it sends an READB256 or READB512 message to it, specifying which block it wants how does the filesystem send this message? Is it an IPC? yes ipc ok. :) so upon init, the filesystem gets the PID of the block driver it's supposed to use? upon mount you mean? right [ nate joins ] nah it just does an open("/dev/device",O_READ); like usual so how does it send the readblock? see a file is just a communication channel in jos Stryyker 51-57:51- Topic 50(36#wgs5650): JOS: Filesystem<->block device comms hrm sendCon(fd, READB256, 100, &buffer); dude. aha ok! sendcon takes care of it. I see yeah everything is ipc read() and write() just send off IO_READ and IO_WRITE messages instead of printf() it does sendcon() right ok good, good. I"m beginning to understand * jbevren works it in his mind its just like qnx basically I"m not familiar with qnx ;) they have some good explanations of the whole thing on their site ill find a good qnx link.. 51-57:51- Topic 50(36#wgs5650): changed by 36jbevren5650: JOS: New block device model we've covered what's being done. let's do some moving of responsibility :) (de)blocking is to be done by the filesystems 256/512 translation et.al why? so block devices will have these IPC's: read write flush query eject upload download direct cause this way, it wont matter what fielsystem you use on what device also, UHS will be dealing with multiple block sizes in many cases giving JOS that ability early will make getting UHS into jos easy as pie so you're saying only use 512 byte blocks on the device side? no :) the query IPC is the key it will return the disk type (3D or 1D), the size of the blocks on the media, whether or not its writeable, online, and removeable the fs driver knows what size of block it needs. it'll compare the driver's block size and set up (de)blocking as needed you could put soci's FS on a fd2000 or ramlink. Mind you, it wont be bootable, but its a price. no i think the device driver should handle that.. ok, different model option then: no specific block size yeah to write, you specify a starting block, instead it should be a SEEK and READ combined into one and how many bytes to write consectuively so only one ipc is needed (or read) now, this DOES need some translation if you want the block device to do it (here it comes) ioctl(fd, IOCTL_SET_BLOCK_SIZE, 256) or 512 1024 it'd be an even power of two. that way the driver can bitshift as approprate :))) lda myblocksize and requestdblocksize hmm id say not have an ioctl for that but just include it in the ipc rotate until one bit hits carry rotate until another :) count the distance and that's how far to shift ;) if it's >16, assume no shift READBLOCK, blocknum, blocksize, length or whatever limit, you get the idea hrm I'd rather not pass blocksize every read that's why I IOCTL'd it the less data we move, the faster we are I'd have to go with jb's idea. yeah i guess (since the filesystem won't change until you remove the disk) (or partition) its just that there is no server (driver) side to a connection in the drivers ive written so far if we were 32 or 40mhz, sure :) so if it changed to that, each connection would need to keep state info about what block size it was nonono I can see this hrm what process is responsible for detecting a disk change? Ok, it is needed. or rather, what part of the communication chan in Nate\DAC: the block driver (of course) well that's probably how I'd do it, but I mean what part is responsible right now (since some devices can't handle a disk change yet) do read and write need to be separate? :) hell, sure. I have 7 IPC's (setsize eliminated) read write flush query eject upload download oh, 8. direct (for sending scsi/ide devices direct commands) generally read and write get their own, but stuff like eject and so on would use ioctl i.e. lowlevel format, cache settings on-dis ioctl flush,eject,direct ? yeah it doesnt really matter k for upload and download, they're debugging commands to send data (firm/software) to a hardware device like a 15x1 oh right you can download sector pointers, etc (that was added during stryyker's discussion) now the real shame is, we cant reset the IEC bus :( we should tie a userport line to the IEC reset :) [ GATOS is a peecee linux project, but the IOCTL definition may be helpful to the readers. ] what exactly is an ioctl() anyway? I see this a lot in the GATOS devel list. ioctl is just a way to do things that arent strictly reading/writing Nate\DAC: IO control.. for anything that doesnt need a full IPC call, or for anything as-yet undefined by the standard interface, but provided by the driver miscallaneous things ok register controls and such for example, yeah kinda what I thought. ok, do continue :) so, read() and write() are as such: foo(fd, blksize, startblock, length); where blksize is an even power of 2 greater than 1 and less than 65536 startblock is a 32bit unsigned INT length is a 32 (24bit?) U-INT id make length a 16bit int (I prefer 32bit as UHS will use it) uint that limits me to the first 65536 sectors :( why would you ever want to read/write 65536 secors at once?! oh sorry! :) Im tired and mistook length for start hehe :) len is an unsigned int and startblock is a long? right nate Ahh, both unsigned yeah (didn't see it up there) jolz: should blksize be a uINT or a byte? uint if byte, we could add flags (nocache) etc jbevren: cacheing should be a filesystem thing.. could all this be put up somewhere? Jbev, send yer log to Greg :) jolz: right. I guess I already established that (blush) he loves this stuff Stryyker: I'll put it up under UHS's site until someone grabs it for the JOS page all I was really interested in was the block device we've covered that so far on the read and write :) now, ioctl: flush isnt needed (no caching) yeah ok theres something that needs thinking about.. ioctl: query will contain a pointer to a malloc()'ed area of memory to which data will be inserted. * jbevren listens actually lets sort the other details first.. ok I"m scratching the ioctl query data layout [pause] ok: the query info will be dumped to the malloc()'ed space and will contain: native block size; number of blocks; geometry (0/0/0 if LBA), and flags: [removeable; software lockable; read-only] anything else needed? add LBA to the flags hmm not in the general case.. ok, theres already a query for iec devices which has iec specific things.. right stuff that under ioctl :) its a driver specific thing yeah for upload and download, upload(fd, deviceID, startmem, size) device ID is needed in case there's mroe than one device (or cpu, whatevver) in the target yeah and no 32 byte limit :) size is a 16bit uint startmem is (paranoia) 32bit uint this is to make it the same size as the read and write calls the same buffer can be used :) normally ipc is on the stack right, but still :) also just so you know, all pointers usually are just 32bit because its easier to not have to switch to 8bit acc or anything yeah so we covered the ipc for block devices. Now, optional extension: we only used a byte for the size field, which leaves 8 bits unused :) I'd enjoy a direct-mem flag, which would make the device driver send the data to a specific location in memory instead of passing it through all the layers ;) hrm. actually, well it can without a flag.. it just uses whatever buffer its given how about (brainstorming): foo(fd, blksize<<24|startram, startblock, size); that's 3 32bit uint's and includes the starting location in mem, the size of block, etc erm 1 int and two longs * jbevren raises his brow well its best just to make it simple, as they normally go on the stack memory size of message isnt an issue, code size to get the message ready should be simplest hrm right but is it that much to ldb #blksize? * jbevren forgets about C programmers though :) ldb? lda:swa xba, whichever its an asm macro I normally use actualy, its xba:lda:xba then you and in the other half but if you dont mind pushing the extra onto the stack, we'll just remember that macro for other things :) yeah, pea 1 looks easier :) foo(sectsize, startmem, statblock, size) ok? yep ok! it'll make it much easier to figure out dma too ;) and its perfec tfor the REU block driver ;) muahahah at any rate. that covers the block device dma is only the first 64k tho isnt it and how filesystems will handle (de)blocking well, xset r rate 250 40 oops until I get done RE'ing the rocket socket, yes >:-) but that's why a device will have an internal DMA mask a mask of 0000FFFF indicates only the first 64k :) 00FFFFFF is 16M (ripped shamelessly from the amiga) but that's a matter of driver structure, and isnt seen outside the driver yeah so its a non-issue speaking of 64k dma limitation how would one specifically malloc() some low mem? :) theres kmalloc() which means kernel, but its really BANK 0 memory ok! as long as its possible do you have a summary scratched down? (I dont) :) a summary of all these ipcs? yeah :) I"m going to get some sleep, and wanted to glance at it I guess I should have launched jot ;) heh jolz: Great discussion :) I hope this gets used :) it will :) Good. It's modelled after my thoughts for UHS's structure :) well basically the only changes to the block device drivers for jos is the new readblock and the query jbevren: something for you to think about for next time.. how will we deal with removable media that contains multiple partitions, such as fd disks? jolz: Good question OH add to flags (It was in my text doc, but I forgot to post it) software_lockable but it shouldnt be hard. You can use an eject command, which might not be able to actually eject the media, but can prep or check it to see if it can be removed well the thing is.. the filesystem asks the device if its changed.. can it be possible to have the ability to get cpu time every second or so? (to check for online/offline changes)