Chapter 7 HPSS User Interface Configuration
442 September 2002 HPSS Installation Guide
Release 4.5, Revision 2
Meetingswith Transarc and IBM Austin have taken place to discuss the issue. The above restriction
may be fixed in the future.
7.6.2.7.2 Migration and Purge Algorithms
Currently,theHDM reads through all the anodes in an aggregate to determine migration and purge
candidates.Using empirical data we determined that the HDM reads approximately 70 entries per
second(this is disk hardware dependent). The optimum number of objects stored on an aggregate
for migration purposes can be determined using the following parameters:
• The amount of time it takes to read through all objects on the aggregate.
• The frequency of migration runs (purge runs when space is needed).
• The amount of time it takes to move files from DFS to HPSS.
Ifmigration runs once per day, and data can be moved in 4 hours, the current suggestion is to allow
approximately500,000 objects on an aggregate. This means that migration will take approximately
6hours (7142 seconds = ~2 hours for header rumble + 4 hours for data movement). Obviously,these
numbers vary dramatically and are dependent on file size and the number of files that need to be
migrated.Remember, only new files and files whose data has been altered need to be migrated into
HPSS.Files that have already been migrated into HPSS, but have been read need to update XDSM
attributes, but this can be done quickly (approximately 60 per second).
Episode XDSM(DMAPI) changes have been discussed between HPSS and Transarc. It is believed
thatwith these changes, the Episode XDSM(DMAPI) rate to read all objects in an aggregate would
be increased to approximately 400 per second. This would greatly reduce the time taken to
determinemigration and purge candidates. Then the suggested limit for the number of objects on
an aggregate could be raised to 2 million objects.
7.6.2.7.3 Fileset Quotas
TheEpisode fileset quota is currently stored in a 32-bit field (actually only 2^31 – 1 can be stored).
The quota represents 1K allocations. So, the current maximum quota per fileset is about 2TB. The
Episodequota reflects the amount of data stored in the fileset plus the amount of anode space. The
quotacan not be increased beyond 2TB. When the quota limit is reached the only way to store more
files is to remove old files.
Whenthe HDM purges data from Episode the quota is not affected. The quota reflects all data ever
stored through the DFS interface. If a file is written to a mirrored fileset through the HPSS
interfaces, the quota is not altered (except for anode space) until the data is actually read through
theDFS interface. So, it is possible to write more than 2TB to a mirrored fileset, but only if the data
is written through a non-DFS interface and never read through the DFS interface (This may occur
iflarge data dumps are made directly to HPSS, but the client wishes to use DFS to manage the name
space).
There are no current plans to change the Episode fileset quota implementation.