More Stuff

All off topic conversation held here. Have fun and play nice. =)

Moderators: Bookworm, starkruzr, MrFireDragon, PrettyPrincess, Wapsi

User avatar
Dave
Posts: 7025
Joined: Tue Jul 31, 2012 5:58 pm
Location: Mountain View, CA, USA

Re: More Stuff

Post by Dave » Thu Jan 17, 2019 11:48 am

GlytchMeister wrote:
Thu Jan 17, 2019 2:24 am
Wait just a Skorm-damned minute.

You. Helped invent. The underlying software and firmware. FOR THE CD-ROM? Only one of the most important portable storage mediums for at least several years?

I... I need to go sit down. Holy shit.
Oh, no... nothing so momentous! :oops:

The CD-ROM as a storage medium already existed at the time. There was a filesystem format (ISO 9660) already in use for data storage on it... a fairly complex standard. I had a SCSI CD-ROM drive on my Mac at home at the time. Knowing about CD-ROM helped me get the job, I'm sure. At the time, CD-ROM hadn't been used for videogame consoles... 3DO was the first.

I didn't want to use the ISO 9660 format for the 3DO... it was designed for data archiving, not for high performance as a "live" filesystem. It was overly complex, and it pretty much requires that you read the entire filesystem directory tree into memory and keep it there at all times... not a great choice for a limited-memory game system built using a small microprocessor core. A single bad scratch in the wrong place could leave most of the files inaccessible and useless.

I designed a different filesystem structure that was better suited to the need. Its big innovation was that it could store two more copies ("avatars") of a file or directory at different places in the disc, and at runtime the o/s would automatically read the one which was closest to the current location of the read head. It would automatically mark an avatar as being damaged if it had trouble reading the data due to a scratch or some dirt on the disc, and would automatically retry the read using a different avatar. This improved both performance and reliability.

A lot of games had only a couple of hundred megabytes of content, which meant that the 3DO disc could store redundant copies of almost all of it, and survive some pretty bad damage... scratches, fingerprints, baby spittle, etc.

From the programmers' point of view it was automatic and invisible... no special effort required. You would build a filesystem image with default options (one avatar per file) on an emulator, and then play the game a few times. The emulator would keep track of which data blocks were being read how many times. An analysis script would then figure out which files were the commonly used ones. You would run the filesystem builder a second time, and it would create a filesystem which used up the entire storage capacity of the disc, scattering multiple avatars of the commonly-used files all over the place... some files might have a dozen avatars, from inner edge to outer. That's the image which was then burned to CD-R and sent off to manufacturing.

A guy named Hedley Davis gave me a suggestion which made it even better... constructing a "startup cache" of exactly those files being read when the game first started, so that it could all be streamed into memory with a minimum number of head seeks. His original idea wasn't directly implementable but I figured out a variation which was (and I neglected to see that he got proper public credit for the original idea when I demonstrated the results... a graceless oversight that I still very much regret).

The device driver worked with an industry-standard ATAPI drive... I already understood SCSI sand there's not a lot of difference at the command level.

User avatar
GlytchMeister
Posts: 3727
Joined: Wed Oct 16, 2013 2:52 pm
Location: Central Illinois
Contact:

Re: More Stuff

Post by GlytchMeister » Thu Jan 17, 2019 4:14 pm

Ok there’s a lot here that I can’t claim to understand but from what I’m gathering, you helped make a better way to put data on a disk?

That’s still pretty damn important, especially for, as you say, video games and such at the consumer level (as those aren’t treated with the kind of respect and care as they would in an office full of computer nerds who know better than to place a disk on anything but a reader shiny-side down).
He's mister GlytchMeister, he's mister code
He's mister exploiter, he's mister ones and zeros
They call me GlytchMeister, whatever I touch
Starts to glitch in my clutch!
I'm too much!

User avatar
Bookworm
Posts: 568
Joined: Sun Jul 29, 2012 11:59 pm
Location: Houston, TX
Contact:

Re: More Stuff

Post by Bookworm » Fri Jan 18, 2019 10:21 pm

GlytchMeister wrote:
Thu Jan 17, 2019 4:14 pm
Ok there’s a lot here that I can’t claim to understand but from what I’m gathering, you helped make a better way to put data on a disk?

That’s still pretty damn important, especially for, as you say, video games and such at the consumer level (as those aren’t treated with the kind of respect and care as they would in an office full of computer nerds who know better than to place a disk on anything but a reader shiny-side down).
No, he didn't find a better way to put data on a disk. What he did was determine a way to make the disk both more resistant to common damage types, and to load the game faster, on a system with limited memory and processing power.

Think of it like a box. A regular CD is a box 10x10 (not really, but this is an analogy).

In the bottom left corner of the box, the space is taken up by a list that says what is in the other 99 spaces. That's ISO 9660. No matter what you do, the first thing you have to do with ISO 9660 is rotate the box to where you can grab that list, then hold it in your left hand so you can constantly refer to it. Now you only have your right hand to reach in and grab whatever someone asks for - and you have to keep turning the box so you can have the space closest to you.

In his box, he set four copies of the list, one in each corner. So, no matter what, you're going to be pretty close to one of the lists. You can check the list _then put it back into the space_. He didn't need the full 96 spaces left, so he divided them up into four equal sets of 24 spaces, and duplicated the information in each quarter. That way, no matter where you were, you could look at the closest list, and then grab the closest copy of the data you need.

Also, this means that if one of the 'lists' wasn't readable due a scratch, it would grab the next one in the rotation. If one of the pieces of data wasn't readable, it'd grab a copy from the next 'quadrant'

Then he made it so that what happened as soon as the disk started spinning, it pre-loaded some of the files necessary to actually run the game, as long as it didn't interfere with the underlying system.

Clear as mud?
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.

User avatar
GlytchMeister
Posts: 3727
Joined: Wed Oct 16, 2013 2:52 pm
Location: Central Illinois
Contact:

Re: More Stuff

Post by GlytchMeister » Fri Jan 18, 2019 11:04 pm

Ok, yeah... I think so. Thanks.

It may come as a bit of a surprise to some folks due to the character I write in the fanfic, but I truly only know enough about computers to break them in interesting ways and to do a better job at bullshitting about knowing about them than the average joe :P
He's mister GlytchMeister, he's mister code
He's mister exploiter, he's mister ones and zeros
They call me GlytchMeister, whatever I touch
Starts to glitch in my clutch!
I'm too much!

User avatar
TazManiac
Posts: 3428
Joined: Fri Nov 29, 2013 6:53 pm

Re: More Stuff

Post by TazManiac » Fri Jan 18, 2019 11:13 pm

"Oh, nothing so momentous,...

No, I was doing this other momentous thing instead..."

I grew up in Silicon Valley and parts north,

Sometimes I feel exactly like Forest Gump, history & momentous events swirling all around me.

User avatar
GlytchMeister
Posts: 3727
Joined: Wed Oct 16, 2013 2:52 pm
Location: Central Illinois
Contact:

Re: More Stuff

Post by GlytchMeister » Fri Jan 18, 2019 11:16 pm

TazManiac wrote:
Fri Jan 18, 2019 11:13 pm
"Oh, nothing so momentous,...

No, I was doing this other momentous thing instead...
Right - still bloody impressive.
He's mister GlytchMeister, he's mister code
He's mister exploiter, he's mister ones and zeros
They call me GlytchMeister, whatever I touch
Starts to glitch in my clutch!
I'm too much!

User avatar
Dave
Posts: 7025
Joined: Tue Jul 31, 2012 5:58 pm
Location: Mountain View, CA, USA

Re: More Stuff

Post by Dave » Fri Jan 18, 2019 11:30 pm

Bookworm wrote:
Fri Jan 18, 2019 10:21 pm
Also, this means that if one of the 'lists' wasn't readable due a scratch, it would grab the next one in the rotation. If one of the pieces of data wasn't readable, it'd grab a copy from the next 'quadrant'

Then he made it so that what happened as soon as the disk started spinning, it pre-loaded some of the files necessary to actually run the game, as long as it didn't interfere with the underlying system.

Clear as mud?
That's basically correct. It was a bit more general than that. Not only could it store several copies of the "list of what it's where", it would automatically store two or more copies of each of the "things" themselves, if there was space enough. If the game had a file of music that was being read in and played, there might be five or six copies of it scattered around in the "storage box". Any one of the "lists" would tell you where all of the identical copies of the music file were located.

This was one of the two "bright idea" I've had in my career that I'm significantly proud of.

The other was figuring out that I could find prime numbers with a hidden message buried inside, and use these as part of a cyber security and encryption system.

User avatar
Bookworm
Posts: 568
Joined: Sun Jul 29, 2012 11:59 pm
Location: Houston, TX
Contact:

Re: More Stuff

Post by Bookworm » Sat Jan 19, 2019 11:41 am

Dave wrote:
Fri Jan 18, 2019 11:30 pm
Bookworm wrote:
Fri Jan 18, 2019 10:21 pm
Also, this means that if one of the 'lists' wasn't readable due a scratch, it would grab the next one in the rotation. If one of the pieces of data wasn't readable, it'd grab a copy from the next 'quadrant'

Then he made it so that what happened as soon as the disk started spinning, it pre-loaded some of the files necessary to actually run the game, as long as it didn't interfere with the underlying system.

Clear as mud?
That's basically correct. It was a bit more general than that. Not only could it store several copies of the "list of what it's where", it would automatically store two or more copies of each of the "things" themselves, if there was space enough. If the game had a file of music that was being read in and played, there might be five or six copies of it scattered around in the "storage box". Any one of the "lists" would tell you where all of the identical copies of the music file were located.

This was one of the two "bright idea" I've had in my career that I'm significantly proud of.

The other was figuring out that I could find prime numbers with a hidden message buried inside, and use these as part of a cyber security and encryption system.
Steganography? (and that's why I suggested four copies of everything in my rough analogy. I know it would be less or more depending on file importance and frequency of access).
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.

User avatar
Dave
Posts: 7025
Joined: Tue Jul 31, 2012 5:58 pm
Location: Mountain View, CA, USA

Re: More Stuff

Post by Dave » Sat Jan 19, 2019 1:35 pm

Bookworm wrote:
Sat Jan 19, 2019 11:41 am
Steganography? (and that's why I suggested four copies of everything in my rough analogy. I know it would be less or more depending on file importance and frequency of access).
No... two separate developments for two companies. The 3DO files were all in the clear... nothing hidden, just replicated.

The prime-number gimmick was when I was developing a public-key cryptography system for TiVo. The algorithm I was using was ElGamal (RSA was still patented at the time). ElGamal public and private keys include a large prime number, so I had to find a few.

Everybody finds big prime numbers in pretty much the same way... generate a large odd number at random and test it to see if it's prime... if not, generate another large number and try again. My brainflash was the realization that I could select candidate numbers that were only mostly random; some of the bits in the center could be set to a predictable, chosen pattern. This wouldn't affect the quality of the search or the properties of the resulting prime numbers at all, and the public-key algorithms work just fine.

So, what I ended up with, is prime numbers that look quite ordinary when you print them out in decimal. But, when you look at them in memory or on disk, when they're stored in the usual Intel format (least significant byte first), the middle part of the number can be read as a printable string in ASCII. The same bits can be interpreted in two different ways... as part of a big number or as a human-readable message.

You've probably seen https://en.m.wikipedia.org/wiki/The_Treachery_of_Images - the painting of a pipe, which tells you that it isn't a pipe? I did the opposite. Pretty much every first-generation TiVo digital video recorder has a public key, which contains a prime number, which contains an embedded string that reads "This prime number was brought to you by (my name) at TiVo Inc."

Later, I needed an even larger number, to use in the "software signing" keys. For this key, I made a prime which contains the Monty Python "Funniest Joke In The World". Their skit claims that if you read and understand the joke (it's written in a gibberish pseudo-German) you die laughing. So, it's a key which defends itself... try to hack it and the Joker gets you! ;)

And, to close the circle in a delightful way... a few months ago I had the opportunity to explain this use of the ElGamal public-key system, to Dr. Tarick ElGamal himself... he did a talk on next-generation Internet security issues that I was able to attend, and I spoke with him afterwards. He seemed amused.

User avatar
Atomic
Posts: 2679
Joined: Tue Jul 31, 2012 12:39 am
Location: Central PA
Contact:

Re: More Stuff

Post by Atomic » Sat Jan 19, 2019 6:04 pm

Dave wrote:
Sat Jan 19, 2019 1:35 pm
Later, I needed an even larger number, to use in the "software signing" keys. For this key, I made a prime which contains the Monty Python "Funniest Joke In The World". Their skit claims that if you read and understand the joke (it's written in a gibberish pseudo-German) you die laughing. So, it's a key which defends itself... try to hack it and the Joker gets you! ;)
Image
Don't let other peoples limitations become your constraints!

My Deviant Art scribbles
The Atomic Guide to Basic GIMP Stuff

User avatar
Hansontoons
Posts: 954
Joined: Tue Jul 31, 2012 9:22 pm
Location: Houston, TX

Re: More Stuff

Post by Hansontoons » Sat Jan 19, 2019 9:04 pm

Wenn ist das Nunstück git und Slotermeyer? Ja! Beiherhund das Oder die Flipperwaldt gersput!

Jasper still has her nose.
IMG_4214.JPG
IMG_4214.JPG (78.46 KiB) Viewed 2046 times

User avatar
Dave
Posts: 7025
Joined: Tue Jul 31, 2012 5:58 pm
Location: Mountain View, CA, USA

Re: More Stuff

Post by Dave » Sun Jan 20, 2019 2:01 am

Hansontoons wrote:
Sat Jan 19, 2019 9:04 pm
Jasper still has her nose.
Possibly because she is not a German Shepherd?

Alkarii
Posts: 1504
Joined: Sun Nov 09, 2014 3:02 pm

Re: More Stuff

Post by Alkarii » Sun Jan 20, 2019 8:38 am

My dad had jury duty Thursday. 13 hours, and the defendant got 160 years and a $134,000 fine, though if he behaves himself, he might get out when he's about 60 or so. Not sure if it was child porn or what, but the guy was a pedophile, apparently.
There is no such thing as a science experiment gone wrong.

User avatar
Dave
Posts: 7025
Joined: Tue Jul 31, 2012 5:58 pm
Location: Mountain View, CA, USA

Re: More Stuff

Post by Dave » Sun Jan 20, 2019 11:57 am

Alkarii wrote:
Sun Jan 20, 2019 8:38 am
My dad had jury duty Thursday. 13 hours, and the defendant got 160 years and a $134,000 fine, though if he behaves himself, he might get out when he's about 60 or so. Not sure if it was child porn or what, but the guy was a pedophile, apparently.
Kudos to your father for being willing to serve... a lot of people will make up any possible excuse to avoid jury duty.

User avatar
Bookworm
Posts: 568
Joined: Sun Jul 29, 2012 11:59 pm
Location: Houston, TX
Contact:

Re: More Stuff

Post by Bookworm » Mon Jan 21, 2019 2:15 am

Dave wrote:
Sun Jan 20, 2019 11:57 am
Alkarii wrote:
Sun Jan 20, 2019 8:38 am
My dad had jury duty Thursday. 13 hours, and the defendant got 160 years and a $134,000 fine, though if he behaves himself, he might get out when he's about 60 or so. Not sure if it was child porn or what, but the guy was a pedophile, apparently.
Kudos to your father for being willing to serve... a lot of people will make up any possible excuse to avoid jury duty.
My problem with Jury duty is simple. 1) I'm self employed. 2) I have a wife and child at home that I have to feed. 3) They keep insisting on making trials longer than they need to be, especially for simple cases. So, if you can find some way to keep my family fed, and my rent paid, for X number of days, and I'll be happy to serve. Otherwise, I show up, tell them my issues, and sit down. If it's a one day trial, I can find a way to do it, because it _is_ important.

I just don't think that racking up a DA's conviction record is more important than my family.
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.

Typeminer
Posts: 648
Joined: Tue Jul 31, 2012 9:34 pm
Location: Pennsylbama, between Philly and Pittsburgh

Re: More Stuff

Post by Typeminer » Mon Jan 21, 2019 1:07 pm

GlytchMeister wrote:
Fri Jan 18, 2019 11:04 pm
Ok, yeah... I think so. Thanks.

It may come as a bit of a surprise to some folks due to the character I write in the fanfic, but I truly only know enough about computers to break them in interesting ways and to do a better job at bullshitting about knowing about them than the average joe :P
I've been using work computers for almost 35 years. And I can totally relate to that. :mrgreen:
Saccharomyces cerevisiae is the linchpin of civilization.

User avatar
AnotherFairportfan
Posts: 5844
Joined: Thu May 01, 2014 2:53 pm

Re: More Stuff

Post by AnotherFairportfan » Tue Jan 22, 2019 10:44 pm

Bookworm wrote:
Thu Jan 17, 2019 1:05 am
Man, I don't have stories like that. I've spent 25 years in IT, and all I can say is that I've kept a lot of small businesses running that would have otherwise tanked.
I know {slightly} Dennis Hayes ... AND the guy who did most of the hardware design on the original Hayes modems.

I worked for two years at AMI.

Joe Celko, a long-time friend, worked on the SQL standards.

I don't have any really good stories - well, i have several Celko stories, but none of them are particularly related to computers...
Proof Positive the world is not flat: If it were, cats would have pushed everything off the edge by now.

User avatar
TazManiac
Posts: 3428
Joined: Fri Nov 29, 2013 6:53 pm

Re: More Stuff

Post by TazManiac » Wed Jan 23, 2019 10:54 pm

Standing on the Quad (Not Stanford, Levi Strauss & Co*), waiting for Steve Jobs to finish talking with a high end Exec (I think it was a Haas)- I'm pretty sure my fan-boi asking for the autograph had something to do with whether or not the negotiations went through (cause I'm big-headed like that :P ) but prob. not...

But they (LS&Co) did deploy the Compaq 396s in place of desktop mainframe terminals, complete w/ Connor Hard Drives which eventually began to stick when cold-starting. You could literally open the case and with the freed-up drive in your hand, torque it and get the platters spinning. great fun.

OTOH, that model was the first mass market entry w/ the 386 chip in it, successor to IBM's (well, Intel's) IBM AT architecture.

Compaq was a hell of an upstart, way back when...

User avatar
Bookworm
Posts: 568
Joined: Sun Jul 29, 2012 11:59 pm
Location: Houston, TX
Contact:

Re: More Stuff

Post by Bookworm » Thu Jan 24, 2019 10:24 pm

TazManiac wrote:
Wed Jan 23, 2019 10:54 pm
Standing on the Quad (Not Stanford, Levi Strauss & Co*), waiting for Steve Jobs to finish talking with a high end Exec (I think it was a Haas)- I'm pretty sure my fan-boi asking for the autograph had something to do with whether or not the negotiations went through (cause I'm big-headed like that :P ) but prob. not...

But they (LS&Co) did deploy the Compaq 396s in place of desktop mainframe terminals, complete w/ Connor Hard Drives which eventually began to stick when cold-starting. You could literally open the case and with the freed-up drive in your hand, torque it and get the platters spinning. great fun.

OTOH, that model was the first mass market entry w/ the 386 chip in it, successor to IBM's (well, Intel's) IBM AT architecture.

Compaq was a hell of an upstart, way back when...
While at university (1990), one guy had SCSI drives that were salvaged in his system. They suffered from stickification. So what he would do is turn on the power button, then hit the side of the case with his hand (or foot). That would jolt the platters loose and they'd start to spin.
I'll get a life when it's proven and substantiated to be better than what I'm currently experiencing.

User avatar
jwhouk
Posts: 5856
Joined: Wed Aug 01, 2012 7:58 am
Location: The Valley of the Sun, Arizona
Contact:

Re: More Stuff

Post by jwhouk » Thu Jan 24, 2019 10:31 pm

Percussive Maintenance. I've heard about that.
"Character is what you are in the dark." - D.L. Moody
"You should never run from the voices in your head. That's how you give them power." - Jin

Post Reply