# System overhead on external HD



## Snydley (Mar 30, 2007)

I just connected a Samsung 1T SATA hard drive to my VIP622 via a Coolmax CD-391 HD enclosure, my VIP622 saw it, formatted it, and when everything was said and done my the receiver reported that I had 930.3G of space available on the drive. Is this normal? I understand that the receiver may use part of the drive for system files etc., but 69.7G seems like alot of space to be used by the system. Anyone seen anything like this on their setup? Anyone have any idea where all that space went? Is there anyway to recover it?
Thanks,
Snyde


----------



## ChuckA (Feb 7, 2006)

Sounds like unrecoverable overhead to me. The drive will be formatted to a LINUX filesystem format and so it will have some amount of fs overhead. I don't have a drive that large to compare but it seems reasonable for a 1T drive. Its only 7% of the space. It looks like my Windows 250g external drive formatted to NTFS fs takes about 9% for formatting overhead.


----------



## Stewart Vernon (Jan 7, 2005)

This has mostly nothing to do with the Dish receiver. Put that same drive on a Windows machine and format it NTFS and you'll pretty much get the same results.

I do wonder and admit it seems excessive too... but the key thing is that there is increasing overhead required to keep track of a larger hard drive. In the olden days we used to talk about FAT (file allocation table) which was data written to the drive that helped the computer keep track of where files were stored on that drive. Similar stuff exists today in the different file formats, and requires a lot of data to keep track of the info on the hard drive.

Still, it admittedly does seem odd to have 70GB of unusable space on a 1TB drive when many of us 5 or so years ago didn't even have a 70GB drive in our computers!


----------



## nicktripp (Dec 1, 2008)

It's actually not drive or operating system overhead. It's a difference in how hard drive manufacturers account for space versus how they're actually calculated by computers. Basically, they advertise drive space calculated in base 10 numbers when they're actually calculated using binary math by computers.

I will not do an adequate job of trying to explain that, so I'll just post a link instead. 

http://compreviews.about.com/od/storage/a/ActualHDSizes.htm

Essentially, drive manufacturers tried to simplify things for consumers when they should have left well enough alone. As drive capacity has increased, this has lead to much confusion.


----------



## ChuckA (Feb 7, 2006)

Well, it is true that the drive makers use 1000G to mean 1T instead of 1024G but that is not the entire difference. The filesystem format does use disk space for managment purposes and therefore it reduces the amount of available space. And different filesystems use different amounts of overhead. In any case, all things considered, I think what you got is what you have and there is nothing you can do to increase or decrease the space available on that drive.


----------



## BattleZone (Nov 13, 2007)

My new 1.5 TB disk formatted NTFS to give 1.38 TB of free space. It's normal overhead.


----------



## nicktripp (Dec 1, 2008)

ChuckA said:


> The filesystem format does use disk space for managment purposes and therefore it reduces the amount of available space. And different filesystems use different amounts of overhead.


This is true, but what you're talking about are partition tables and the boot sector, both of which take up a minimal amount of space on any hard drive. Less than 1MB for any file systems any of us would be using.

The difference in space mentioned between a "1TB" drive and the 930.3GB of usable space reported by the OS can be entirely attributed to the marketing math I mentioned. To prove it, I'll show you the math.

Drive manufacturers state that 1KB = 1,000 Bytes and 1MB = 1,000KB = 1,000,000 Bytes, etc.

In reality 1KB = 1,024 bytes and 1MB = 1,024KB = 1,048,576 Bytes.

If you push this to the 1TB level where a drive manufacturer states that 1TB = 1,000GB there is a very large discrepancy which we can see by looking at that extrapolated out.

According to drive manufacturers...

1KB = 1,000 Bytes
1MB = 1,000,000 Bytes
1GB = 1,000,000,000 Bytes
1TB = 1,000,000,000,000 Bytes

In reality...

1KB = 1,024 Bytes.
1MB = 1,048,576 Bytes
1GB = 1,073,741,824 Bytes
1TB = 1,099,511,627,776 Bytes

Do you see the discrepancy between the listed capacity and how a TB would actually be calculated? It's 99,511,627,776 Bytes.

They didn't sell you a 1TB hard drive. They sold you a 1,000,000,000,000 Byte hard drive. 1,000,000,000,000 Bytes = 931.322575 Gigabytes

So you see, the marketing math problem does actually explain the discrepancy quite well.


----------



## SaltiDawg (Aug 30, 2004)

HDMe said:


> ...
> Still, it admittedly does seem odd to have 70GB of unusable space on a 1TB drive when many of us 5 or so years ago didn't even have a 70GB drive in our computers!


I still don't have 70GB. :hurah: (My first three computers didn't even have a hard drive. lol )


----------



## Stewart Vernon (Jan 7, 2005)

nicktripp said:


> They didn't sell you a 1TB hard drive. They sold you a 1,000,000,000,000 Byte hard drive. 1,000,000,000,000 Bytes = 931.322575 Gigabytes


I'm not sure how you got to that number... When I take 1,000,000,000,000 and divide it by 1024 twice, I come up with 953 GB (plus some fractions)... So even if they are using the fuzzy math, it still means 20 GB or so is being used by the allocation tables and such.


----------



## Snydley (Mar 30, 2007)

nicktripp said:


> This is true, but what you're talking about are partition tables and the boot sector, both of which take up a minimal amount of space on any hard drive. Less than 1MB for any file systems any of us would be using.
> 
> The difference in space mentioned between a "1TB" drive and the 930.3GB of usable space reported by the OS can be entirely attributed to the marketing math I mentioned. To prove it, I'll show you the math.
> 
> ...


I was wondering about that myself,(the math thing), I didn't realize it would come out to be SO large though.
Well anyway, that explains it. If that's what I've got, then so be it. I've still got more than the 500G drive it replaced. Thanks for the help EVERYONE!
Snyde


----------



## nicktripp (Dec 1, 2008)

HDMe said:


> I'm not sure how you got to that number... When I take 1,000,000,000,000 and divide it by 1024 twice, I come up with 953 GB (plus some fractions)...


You need to divide by 1,024 a third time. By dividing twice you are looking at MB, not GB. Divide a third time and you'll see 1,000,000,000,000 bytes represented as gigabytes.

1,000,000,000,000 bytes = 9,76,562,500 kilobytes

9,76,562,500 kilobytes = 953,674.31640625 megabytes

953,674.31640625 megabytes = 931.32257461547852 gigabytes

931.32257461547852 gigabytes = 0.909494701772928 terabytes

Ta da! :icon_da:


----------



## Stewart Vernon (Jan 7, 2005)

nicktripp said:


> You need to divide by 1,024 a third time. By dividing twice you are looking at MB, not GB. Divide a third time and you'll see 1,000,000,000,000 bytes represented as gigabytes.
> 
> 1,000,000,000,000 bytes = 9,76,562,500 kilobytes
> 
> ...


My bad... That's what I get for working with a handheld calculator where I can't punch in that high of a number without using scientific notation... so I had started from 1,000 lower to begin with and didn't divide the extra 1024.

So, yeah... if they are using the fuzzy math (and I suspect now that you are 100% correct after all) then this is right in-line.

It's one of those situations where even though I know the difference, I was still being confused. I really wish they would have just enforced the 1024 byte-per-KB measure from the beginning and forced everyone to learn it. Then we wouldn't have all the confusion.

I guess there isn't much overhead after all it would seem. Thanks for setting me straight where I messed up!


----------



## TulsaOK (Feb 24, 2004)

SaltiDawg said:


> I still don't have 70GB. :hurah: (My first three computers didn't even have a hard drive. lol )


I bought my first hard drive for my Compaq 8086; 20 mb for $625. A little more cost effective nowadays.


----------



## bartendress (Oct 8, 2007)

TulsaOK said:


> I bought my first hard drive for my Compaq 8086; 20 mb for $625. A little more cost effective nowadays.


For the life of me, I can't recall how many MG I had in my first Compaq machine, but I clearly remember the cutting edge 2400 baud modem that came with it! My, how times have changed. LOL


----------



## Slordak (Dec 17, 2003)

Again, to reiterate what was said above, this is "bad math" from the drive manufacturers and has nothing to do with any specific drive or to Dish Network's treatment of the drive. Unfortunately, this same "well, 1000 bytes is sort of a kilobyte" concept has now spilled over into flash drives, where one can buy a USB flash drive of "2 GB" capacity and find that it's actually... not.

Monitor manufacturer's did this sort of thing a while back, where they would quote the size of the screen including the frame/bezel (which was quite large on some CRT monitors). Hence, they would describe a monitor's size as, say, "17 inch screen, 15.6 viewable", which would mean that it was really a 15.6 inch monitor. After some lawsuits and a shift to LCD panels, they are happily no longer doing this.


----------



## SaltiDawg (Aug 30, 2004)

Slordak said:


> ... Hence, they would describe a monitor's size as, say, "17 inch screen, 15.6 viewable", which would mean that it was really a 15.6 inch monitor. After some lawsuits and a shift to LCD panels, they are happily no longer doing this.


Whoa! Do you mean it's illegal to exaggerate about size?


----------



## tnsprin (Mar 16, 2003)

Slordak said:


> Again, to reiterate what was said above, this is "bad math" from the drive manufacturers and has nothing to do with any specific drive or to Dish Network's treatment of the drive. Unfortunately, this same "well, 1000 bytes is sort of a kilobyte" concept has now spilled over into flash drives, where one can buy a USB flash drive of "2 GB" capacity and find that it's actually... not.
> 
> Monitor manufacturer's did this sort of thing a while back, where they would quote the size of the screen including the frame/bezel (which was quite large on some CRT monitors). Hence, they would describe a monitor's size as, say, "17 inch screen, 15.6 viewable", which would mean that it was really a 15.6 inch monitor. After some lawsuits and a shift to LCD panels, they are happily no longer doing this.


Actually its good math by drive manufacturers, but bad math commonly in use in software. It was too easy to talk in as if it 1024 was a k and the number made little difference in the early days when a big mainframe might have a just over a Million bytes of memory. Disk drives were not then using fixed sized clusters on a disk to make up records. Instead files were were in records or blocks (e.g 5 records in a block). The track was then written with as many records as would fit. There were complicate formulas for figuring the extra bytes written with each Block, error checks, and gap between blocks. Printed tables were most often used by programmers to decide on the best blocking factor to choose. And disk drives were large in size and small in capacity. The first disk system I used a lot when I started programming was only 7MB. Even today, mainframes tend to simulate later model disk drive even though they now are actually on drives that are much the same as those in your PC.


----------



## ChuckA (Feb 7, 2006)

Sounds like we have the same background! I still code in assembler every day.


----------



## P Smith (Jul 25, 2002)

Back to the topic.

All 622 or 722 EHD have same overhead - one EXT3 partition 1 GB size, rest of HDD allocated by second ( for 500 GB disks or less ) or second and third EXT3 type partition(s) for your recordings.
No hidden space.


----------



## P Smith (Jul 25, 2002)

Here is partitioning of Samsung 1 TB as 622's EHD:

```
Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
1 heads, 63 sectors/track, 31008336 cylinders
Units = cylinders of 63 * 512 = 32256 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               2       33298     1048855+  83  Linux
/dev/sdb2           33299    16677334   524287134   83  Linux
/dev/sdb3        16677335    31008336   451426563   83  Linux


Disk /dev/sdb: 1 heads, 63 sectors, 31008336 cylinders

Nr AF  Hd Sec  Cyl  Hd Sec  Cyl     Start      Size ID
 1 00   0   1    1   0  63 1023         63    2097711 83
 2 00   0  63 1023   0  63 1023    2097774 1048574268 83
 3 00   0  63 1023   0  63 1023 1050672042  902853126 83
 4 00   0   0    0   0   0    0          0          0 00
```
And 2 TB (WD20EARS)


```
Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
1 heads, 63 sectors/track, 62016336 cylinders
Units = cylinders of 63 * 512 = 32256 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               2       33290     1048576   83  Linux
/dev/sdb2           33290    16677353   524288000   83  Linux
/dev/sdb3        16677353    33321417   524288000   83  Linux
/dev/sdb4        33321417    62016336   903889974+  85  Linux extended
/dev/sdb5        33321418    49965481   524288000   83  Linux
/dev/sdb6        49965482    62016336   379601911   83  Linux

Disk /dev/sdb: 1 heads, 63 sectors, 62016336 cylinders

Nr AF  Hd Sec  Cyl  Hd Sec  Cyl     Start      Size ID
 1 00   0   0    0   0   0    0         63    2097152 83
 2 00   0   0    0   0   0    0    2097216 1048576000 83
 3 00   0   0    0   0   0    0 1050673217 1048576000 83
 4 00   0   0    0   0   0    0 2099249218 1807779949 85
 5 00   0   0    0   0   0    0         63 1048576000 83
 6 00   0   0    0   0   0    0         63  759203822 83
```


----------



## bartendress (Oct 8, 2007)

P Smith said:


> Here is partitioning of Samsung 1 TB as 622's EHD:
> 
> ```
> Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
> ...


What a mess


----------



## P Smith (Jul 25, 2002)

bartendress said:


> What a mess


What?! Can't read ?


----------



## ehb224 (Apr 4, 2008)

Pretty straightforward. It means that the 622/722 creates 3 primary linux partitions on the drive with the second and third partitions being close to equal in size and the first one being smaller. 


All three partitions are type 83 which is a standard linux filesystem partition. (ext2/ext3)


----------



## bartendress (Oct 8, 2007)

P Smith said:


> What?! Can't read ?


I can read perfectly fine. Thanks for asking.


----------



## TulsaOK (Feb 24, 2004)

bartendress said:


> I can read perfectly fine. Thanks for asking.


Speaking of pushing something down a flight of stairs.....


----------



## P Smith (Jul 25, 2002)

ehb224 said:


> Pretty straightforward. It means that the 622/722 creates 3 primary linux partitions on the drive with the second and third partitions being close to equal in size and the first one being smaller.
> 
> All three partitions are type 83 which is a standard linux filesystem partition. (ext2/ext3)


I would guess developers of the EHD didn't had that time a disks bigger then 500 GB, so they did tests on 0.5 TB disks first and made allocation by max size, then continued on HW RAID-0 2x500 GB and split space but stick with first plan.


----------



## TulsaOK (Feb 24, 2004)

P Smith said:


> I would guess developers of the EHD didn't had that time a disks bigger then 500 GB, so they did tests on 0.5 TB disks first and made allocation by max size, then continued on HW RAID-0 2x500 GB and split space but stick with first plan.


Well said, P.


----------



## bartendress (Oct 8, 2007)

TulsaOK said:


> Speaking of pushing something down a flight of stairs.....


:goodjob:


----------



## P Smith (Jul 25, 2002)

Add partition's info for 722/622's EHD 2 TB size to the post#20.

As I mentioned a few times (regardless of all the skeptical posts  ), dish developers still apologetic of 500 GB partitions. Duh !


----------



## bnborg (Jun 3, 2005)

SaltiDawg said:


> I still don't have 70GB. :hurah: (My first three computers didn't even have a hard drive. lol )


I don't count my Commodore 64's.

My first real computer (386x16) had a 40 MB hard drive.


----------

