Intelligent data storage manager Number:7,114,013 from the United States Patent and Trademark Office (PTO) owispatent The intelligent data storage manager functions to combine the non-homogeneous physical devices contained in a data storage subsystem to create a logical device with new and unique quality of service characteristics that satisfy the criteria for the policies appropriate for the present data object. In particular, if there is presently no logical device that is appropriate for use in storing the present data object, the intelligent data storage manager defines a new logical device using existing physical and/or logical device definitions as component building blocks to provide the appropriate characteristics to satisfy the policy requirements. The intelligent data storage manager uses weighted values that are assigned to each of the presently defined logical devices to produce a best fit solution to the requested policies in an n-dimensional best fit matching algorithm. The resulting logical device definition is then implemented by dynamically interconnecting the logical devices that were used as the components of the newly defined logical device to store the data object.<H Intelligent, manager, Intelligent, dat, 7114013.html, Patent, pto, 7114013

Title: Intelligent data storage manager

Abstract: The intelligent data storage manager functions to combine the non-homogeneous physical devices contained in a data storage subsystem to create a logical device with new and unique quality of service characteristics that satisfy the criteria for the policies appropriate for the present data object. In particular, if there is presently no logical device that is appropriate for use in storing the present data object, the intelligent data storage manager defines a new logical device using existing physical and/or logical device definitions as component building blocks to provide the appropriate characteristics to satisfy the policy requirements. The intelligent data storage manager uses weighted values that are assigned to each of the presently defined logical devices to produce a best fit solution to the requested policies in an n-dimensional best fit matching algorithm. The resulting logical device definition is then implemented by dynamically interconnecting the logical devices that were used as the components of the newly defined logical device to store the data object.

Patent Number: 7,114,013 Issued on 09/26/2006 to Bakke, et al.

Inventors: Bakke; Mark Alan (Maple Grove, MN), Blendermann; Stephen Henry (Boulder, CO), Halladay; Steven (Superior, CO), Johnson; Steven A. (Westminster, CO), Laughlin; Jeffrey Steven (Nederland, CO), Rudeseal; George Arthur (Boulder, CO), Schafer; Charles Patrick (Louisville, CO), West; Thomas David (Boulder, CO), Wojnar; Michael C. (Lafayette, CO)
Assignee: Storage Technology Corporation (Louisville, CO)

Appl. No.: 09/966,263

Filed: September 28, 2001

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
09232431	Jan., 1999	6330621

Current U.S. Class: 710/5 ; 710/200; 710/36; 711/114

Current International Class: G06F 12/00 (20060101)

Field of Search: 710/5,36,200 711/100,114 379/219

References Cited [Referenced By]

U.S. Patent Documents


3130387	April 1964	Wright et al.
3699533	October 1972	Hunter
3806888	April 1974	Brickman et al.
3909799	September 1975	Recks et al.
3949377	April 1976	O'Neill, Jr.
3976977	August 1976	Porter et al.
4021782	May 1977	Hoerning
4040026	August 1977	Gernelle
4054951	October 1977	Jackson et al.
4080651	March 1978	Cronshaw et al.
4080652	March 1978	Cronshaw et al.
4084234	April 1978	Calle et al.
4086629	April 1978	Desyllas et al.
4110823	August 1978	Cronshaw et al.
4123795	October 1978	Dean, Jr. et al.
4126893	November 1978	Cronshaw et al.
4126894	November 1978	Cronshaw et al.
4158235	June 1979	Call et al.
4215400	July 1980	Denko
4228501	October 1980	Frissell
4241420	December 1980	Fish et al.
4246637	January 1981	Brown et al.
4276595	June 1981	Brereton et al.
4298932	November 1981	Sams
4310883	January 1982	Clifton et al.
4318184	March 1982	Millett et al.
4327408	April 1982	Frissell et al.
4412285	October 1983	Neches et al.
4414644	November 1983	Tayler
4533995	August 1985	Christian et al.
4945429	July 1990	Munro et al.
4974156	November 1990	Harding et al.
5131087	July 1992	Warr
5164909	November 1992	Leonhardt et al.
5214768	May 1993	Martin et al.
5303214	April 1994	Kulakowski et al.
5386516	January 1995	Monahan et al.
5388260	February 1995	Monahan et al.
5412791	May 1995	Martin et al.
5430855	July 1995	Walsh et al.
5455926	October 1995	Keele et al.
5475817	December 1995	Waldo et al.
5504873	April 1996	Martin et al.
5506986	April 1996	Healy
5535322	July 1996	Hecht
5537585	July 1996	Blickenstaff et al.
5546557	August 1996	Allen et al.
5560040	September 1996	Mizumachi
5566331	October 1996	Irwin, Jr. et al.
5613154	March 1997	Burke et al.
5619690	April 1997	Matsumani et al.
5625405	April 1997	DuLac et al.
5630067	May 1997	Kindell et al.
5640510	June 1997	Hanaoka et al.
5664186	September 1997	Bennett et al.
5671439	September 1997	Klein et al.
5689481	November 1997	Tamura et al.
5694550	December 1997	Takeda et al.
5710549	January 1998	Horst et al.
5740362	April 1998	Buickel et al.
5751715	May 1998	Chan et al.
5758050	May 1998	Brady et al.
5758085	May 1998	Koucheris et al.
5758125	May 1998	Misinai et al.
5802258	September 1998	Chen
5805864	September 1998	Carlson et al.
5809285	September 1998	Hilland
5828836	October 1998	Westwick et al.
5829046	October 1998	Tzelnic et al.
5829053	October 1998	Smith et al.
5832527	November 1998	Kawaguchi
5838891	November 1998	Mizuno et al.
5845147	December 1998	Vishlitzky et al.
5867648	February 1999	Foth et al.
5881311	March 1999	Woods
5884046	March 1999	Antonov
5890203	March 1999	Aoki
5960451	September 1999	Voigt et al.
5963971	October 1999	Fosler et al.
5996024	November 1999	Blumenau
6111944	August 2000	Molin
6128717	October 2000	Harrison et al.
6141729	October 2000	Ishida et al.
6457139	September 2002	D'Errico et al.

Foreign Patent Documents


892798	Feb., 1972	CA
907211	Aug., 1972	CA
0 535 922	Apr., 1993	EP
0 689 125	Dec., 1995	EP
1167762	Oct., 1969	GB
1353770	May., 1974	GB
1359662	Oct., 1974	GB
1496779	Jan., 1978	GB
1496780	Jan., 1978	GB
1547381	Jun., 1979	GB
2063532	Jun., 1981	GB
51-18409	Feb., 1976	JP
52-106641	Sep., 1977	JP
53-22331	Mar., 1978	JP
53-84632	Jul., 1978	JP
53-98741	Aug., 1978	JP
53-108747	Sep., 1978	JP
55-153058	Nov., 1980	JP
55-164958	Dec., 1980	JP
4-48250	Jun., 1992	JP
98 40810	Jul., 1992	WO
97 07461	Feb., 1997	WO
98 33113	Jul., 1998	WO

Other References

Richard V. Keele, "Optical Storage: Terabytes On-Line for IBM Mainframes", Data/Ware Development, Inc., pp. 68-71, 1988. cited by other .
A.L. Busk et al., "The Storage Server as Virtual Volume Manager", IEEE Symposium on Mass Storage Systems, pp. 79-86, 1993. cited by other .
Sam Coleman, Steve Miller, "Mass Storage System Reference Model", Version 4, Developed by the IEEE Technical Committee on Mass Storage Systems and Technology, 38 pages, May 1990. cited by other .
J.L. Sloan, B.T. O'Lear, D.L. Kitts and B.L. Irwin, MaSSIVE.TM.. The Mass Storage System IV Enterprise. Proceedings of the IEEE. vol. 81, No. 4, pp. 621-630, Apr. 1993. cited by other .
R. Baird, "Virtual Storage Architecture Guide", IEEE Symposium on Mass Storage Systems, pp. 312-326, 1995. cited by other .
Garth A. Gibson, "Network Attached Secure Disks (NASD)", ARPA ITO Scalable Systems and Software, pp. 1-24, May 4, 1997. cited by other .
M. Jander, "Launching Storage-Area Net", Data Communications, US, McGraw Hill, New York, vol. 27, No. 4, Mar. 21, 1998, pp. 64-72. cited by other .
B. Christensen, "Building A Storage-Area Network-SANS Boost Performance, Reliability and Scalability of the Critical Link Between Servers and Storage Devices", Data Communications, US, McGraw Hill, New York, vol. 27, No. 6, Apr. 21, 1998, pp. 67-70, 74. cited by other .
David Nagle, Garth Gibson, Greg Ganger & Bill Courtright. Networking for Network-Attached Storage. NADS/NSIC Networking Workshop, pp. 1-18, Sep. 1, 1998. cited by other .
Shared Virtual Storage System, System Description, 1979,pp. 1-14, Masstor Systems Corporation, Sunnyvale, California. cited by other .
Shared Virtual Storage System, System Overview, 1980, pp. 1-60, Masstor Systems Corporation, Sunnyvale, California. cited by other .
Digest of Papers, The Gap Between MSS Products and User Requirements, Fourth IEEE Symposium on Mass Storage Systems, Apr. 15-17, 1980, pp. 1-76, Nat'l Center for Atmospheric Research, Boulder, Colorado. cited by other .
FPEXEC File Processor Executive Program Logic Manual, Oct. 18, 1982, p. 1-177, MASSTOR Systems Corp., Sunnyvale, California. cited by other .
Waddell, J.M. and Way, D.L., Channel Scheduling By A Control Unit, IBM Technical Disclosure Bulletin, vol. 17, No. 9, Feb. 1975. cited by other .
Liptay, J.S., Structural Aspects of the System/360 Model 85, IBM Systems Journal, pp. 1-14, vol. 7, No. 1, 1968. cited by other .
Meade, Robert M., How a cache memory enhances a computer's performance, Electronics, 1-17-72, pp. 58-63. cited by other .
Smith, Alan Jay, Directions For Memory Hierarchies And Their Components: Research And Development, 1978. cited by other .
Christman, R.D., Collins, M.W., Devaney, M.A., Willbanks, E.W., A Network File Storage System, UC-32, 7-81, pp. 1-14. cited by other .
Reference Manual for IBM 3830 Storage Control Model 1 and IBM 3330 Disk Storage, GA26-1592-5, File No. S360/S370-07, pp. 1-101. cited by other .
Porter, James N., The Disk Drive Industry, IEEE Transactions on Magnetics, vol. MAG-14, No. 4, Jul. 1978, pp. 149-153. cited by other .
Puthoff, S.H., Technical Innovations In Information Storage And Retrieval, IEEE Transactions On Magnetics, vol. MAG-14, No. 4, Jul. 1978, pp. 143-148. cited by other .
Boggs, Jr., Virtual Input/Output channels For A Digital Computer, IBM Tech. Disc. Bull., vol. 20, No. 1, Jun. 1977, pp. 110-112. cited by other .
Nishimukai, T., Tsujioka, S., Kadono, S., Chiba, T., Universal IO Device Controller By Using Programmable Logic Arrays, IEEE 1978, pp. 477-480. cited by other .
Christman, R.D., Collins, M.W., Devaney, M.A., Willbanks, E.W., A Network File-Storage System, Los Alamos Scientific Laboratory, LA-8887-MS, pp. 1-14. cited by other .
Chorn, G.E., Christman, R.D., Klingner, C.T., The Standard File Transport Protocol, LA-7388-MS Informal Report UC-32 Issued: Aug. 1978. cited by other .
Mitchell, Jr., M.J., Diskette Load/Dump Control, IBM Tech. Disc. Bull. vol. 20, No. 4, Sep. 1977, pp. 1359-1361. cited by other .
Collins, M.W., Mass Storage at LASL, Los Alamos Scientific Laboratory, pp. 95-112. cited by other .
The IBM Mass Memory System, Ampex Corporation, Sunnyvale, CA, pp. 1-30. cited by other .
Crittendon, Jr., W.C., Operational Characteristics Of The NCAR Mass Storage Device, pp. 48-50. cited by other .
Howie, Jr., H. Robert, More Practical Applications Of Trillion-Bit Mass Storage Systems, Advanced Systems Dept./Ampex Corp., Sunnyvale, CA. cited by other .
Shell Oil Company, Purchase Order No. SH-84378-JLH, Dec. 18, 1978 w/Agreement Between Masstor Systems Corporation and Shell Oil Co. as an attachment. cited by other .
Masstor, MC Series Computer, Specifications. cited by other .
Blood, P.M., Christman, R.D., Collins, M.W., Willbanks, E.W., Common File System Specifications, LA-7065-MS, Informal Report, 09-77, pp. 1-15. cite- d by other .
IBM 3850 Mass Storage System (MSS) Introduction And Preinstallation Planning, GA32-0038-1, File No. S370-07. cited by other .
IBM System/370 Principles of Operation, GA22-7000-8, File No. S370-01. cit- ed by other .
IBM 3850 Mass Storage System (MSS) Principles of Operation: Theory, GA32-0035-1, File No. S370-07. cited by other.

Primary Examiner: Perveen; Rehana
Attorney, Agent or Firm: Bailey; Wayne P.

Parent Case Text

This is a continuation divisional of application(s) Ser. No. 09/232,431 filed on Jan. 15, 1999 now U.S. Pat. No. 6,330,621.

Claims

What is claimed:

1. A data storage manager operational in a data storage subsystem that uses a plurality of data storage elements to store data thereon for a plurality of host processors that are connected to said data storage subsystem, comprising: means for storing a set of logical data storage device definitions that are created from said plurality of data storage elements; means for identifying a set of data storage characteristics appropriate for a present data object; means for comparing said identified set of data storage characteristics with said stored set of logical data storage device definitions; means, responsive to a failure to match said identified set of data storage characteristics with a single one of said stored set of logical data storage device definitions, for creating a new logical device definition using a plurality of said stored set of logical data storage device definitions; and means for storing said present data object on interconnected ones of said plurality of data storage elements that correspond to said new logical device definition.

2. The data storage manager of claim 1 wherein said means for storing comprises: means for allocating space on an existing instance of said interconnected ones of said plurality of data storage elements that correspond to said new logical device definition.

3. The data storage manager of claim 1 wherein said means for storing comprises: means for creating a new instance of said interconnected ones of said plurality of data storage elements that correspond to said new logical device definition.

4. The data storage manager of claim 1 wherein said means for storing comprises: means for storing data indicative of a plurality of data storage attributes from the class of data storage attributes comprising: speed of access to first byte, level of reliability, cost of storage, probability of recall, and expected data transfer rate.

5. A method of operating a data storage manager operational in a data storage subsystem that uses a plurality of data storage elements to store data thereon for a plurality of host processors that are connected to said data storage subsystem, comprising the steps of: storing a set of logical data storage device definitions that are created from said plurality of data storage elements; identifying a set of data storage characteristics appropriate for a present data object; comparing said identified set of data storage characteristics with said stored set of logical data storage device definitions; creating, in response to a failure to match said identified set of data storage characteristics with a single one of said stored set of logical data storage device definitions, a new logical device definition using a plurality of said stored set of logical data storage device definitions; and storing said present data object on interconnected ones of said plurality of data storage elements that correspond to said new logical device definition.

6. The method of operating a data storage manager of claim 5 wherein said step of storing comprises: allocating space on an existing instance of said interconnected ones of said plurality of data storage elements that correspond to said new logical device definition.

7. The method of operating a data storage manager of claim 5 wherein said step of storing further comprises: creating a new instance of said interconnected ones of said plurality of data storage elements that correspond to said new logical device definition.

8. The method of operating a data storage manager of claim 5 wherein said step of storing comprises: storing data indicative of a plurality of data storage attributes from the class of data storage attributes comprising: speed of access to first byte, level of reliability, cost of storage, probability of recall, and expected data transfer rate.

9. A data storage manager operational in a data storage subsystem that uses a plurality of data storage elements to store data thereon for a plurality of host processors that are coupled to said data storage subsystem, comprising: means for maintaining at least one storage attribute associated with a logical data storage device, said logical data storage device comprising at least a portion of said plurality of data storage elements; means for identifying at least one storage attribute appropriate for a present data object; means for comparing said identified storage attribute with said maintained storage attribute; means, responsive to a failure to match said identified storage attribute with said maintained storage attribute, for creating a new logical data storage device; and means for storing said present data object on that portion of said plurality of data storage elements that correspond to said new logical data storage device.

10. The data storage manager of claim 9, wherein said new logical data storage device comprises both physical and logical data storage elements.

11. A data storage manager operational in a data storage subsystem that uses a plurality of data storage elements to store data thereon for a plurality of host processors that are coupled to said data storage subsystem, comprising: means for maintaining at least one storage attribute associated with a logical data storage device, said logical data storage device comprising at least a portion of said plurality of data storage elements; means for identifying at least one storage attribute appropriate for a present data object; means for comparing said identified storage attribute with said maintained storage attribute; means, responsive to a match of said identified storage attribute with said maintained storage attribute, for creating a new instantiation of said logical data storage device; and means for storing said present data object on that portion of said plurality of data storage elements that correspond to said new instantiation of said logical data storage device.

12. The data storage manager of claim 11, wherein said new instantiation of said logical data storage device comprises both physical and logical data storage elements.

13. In a data storage manager operational in a data storage subsystem that uses a plurality of data storage elements to store data thereon for a plurality of host processors that are coupled to said data storage subsystem, a method comprising the steps of: maintaining at least one storage attribute associated with a logical data storage device, said logical data storage device comprising at least a portion of said plurality of data storage elements; identifying at least one storage attribute appropriate for a present data object; comparing said identified storage attribute with said maintained storage attribute; creating, responsive to a failure to match said identified storage attribute with said maintained storage attribute, a new logical data storage device; and storing said present data object on that portion of said plurality of data storage elements that correspond to said new logical data storage device.

14. A data storage manager operational in a data storage subsystem that uses a plurality of data storage elements to store data thereon for a plurality of host processors that are coupled to said data storage subsystem, comprising: maintaining at least one storage attribute associated with a logical data storage device, said logical storage device comprising at least a portion of said plurality of data storage elements; identifying at least one storage attribute appropriate for a present data object; comparing said identified storage attribute with said maintained storage attribute; creating, responsive to a match of said identified storage attribute with said maintained storage attribute, a new instantiation of said logical data storage device; and storing said present data object on that portion of said plurality of data storage elements that correspond to said new instantiation of said logical data storage device.

15. The data storage manager of claim 1 wherein at least some of the plurality of data storage elements are non-homogeneous physical devices, and portions of a plurality of the non-homogeneous physical devices are logically associated in creating the new logical device definitions.

16. The data storage manager of claim 9 wherein at least some of the plurality of data storage elements are non-homogeneous physical devices, and the new logical data storage device is defined using portions of a plurality of the non-homogeneous physical devices.

17. The data storage manager of claim 11 wherein at least some of the plurality of data storage elements are non-homogeneous physical devices, and the new instantiation of the logical data storage device is defined using portions of a plurality of the non-homogeneous physical devices.

Description

FIELD OF THE INVENTION

This invention relates to data storage subsystems and, in particular, to a dynamically mapped virtual data storage subsystem which includes a data storage manager that functions to combine the non-homogeneous physical devices contained in the data storage subsystem to create a logical device with new and unique quality of service characteristics that satisfy the criteria for the policies appropriate for the present data object.

PROBLEM

It is a problem in the field of data storage subsystems to store the ever increasing volume of application data in an efficient manner, especially in view of the rapid changes in data storage characteristics of the data storage elements that are used to implement the data storage subsystem and the increasingly specific need of the applications that generate the data.

Data storage subsystems traditionally comprised homogeneous collections of data storage elements on which the application data was stored for a plurality of host processors. As the data storage technology changed and a multitude of different types of data storage elements became available, the data storage subsystem changed to comprise subsets of homogeneous collections of data storage elements, so that the application data could be stored on the most appropriate one of the plurality of subsets of data storage elements. Data storage management systems were developed to route the application data to a selected subset of data storage elements and a significant amount of processing was devoted to ascertaining the proper data storage destination for a particular data set in terms of the data storage characteristics of the selected subset of data storage elements. Some systems also migrate data through a hierarchy of data storage elements to account for the timewise variation in the data storage needs of the data sets.

In these data storage subsystems, the quality of service characteristics are determined by the unmodified physical attributes of the data storage elements that are used to populate the data storage subsystem. One exception to this rule is disclosed in U.S. Pat. No. 5,430,855 titled "Disk Drive Array Memory System Using Nonuniform Disk Drives," which discloses a data storage subsystem that uses an array of data storage elements that vary in their data storage characteristics and/or data storage capacity. The data storage manager in this data storage subsystem automatically compensates for any nonuniformity among the disk drives by selecting a set of physical characteristics that define a common data storage element format. However, the data storage utilization of the redundancy groups formed by the data storage manager is less than optimal, since the least common denominator data storage characteristics of the set of disk drives is used as the common disk format. Thus, disk drive whose data storage capacity far exceeds the smallest capacity disk drive in the redundancy group suffers from loss of utilization of its excess data storage capacity. Therefore, most data storage subsystems do not utilize this concept and simply configure multiple redundancy groups, with each redundancy group comprising a homogeneous set of disk drives. A problem with such an approach is that the data storage capacity of the data storage subsystem must increase by the addition of an entire redundancy group. Furthermore, the replacement of a failed disk drive requires the use of a disk drive that matches the characteristics of the remaining disk drives in the redundancy group, unless loss of the excess data storage capacity of the newly added disk drive were incurred, as noted above.

Thus, it is a prevalent problem in data storage subsystems that the introduction of new technology is costly and typically must occur in fairly large increments, occasioned by the need for the data storage subsystem to be comprised of homogeneous subset of data storage devices, even in a virtual data storage subsystem. Therefore, data administrators find it difficult to cost effectively manage the increasing volume of data that is being generated in order to meet the needs of the end users' business. In addition, the rate of technological innovation is accelerating, especially in the area of increases in data storage capacity and the task of incrementally integrating these new solutions into existing data storage subsystems is difficult to achieve.

SOLUTION

The above described problems are solved and a technical advance achieved by the present intelligent data storage manager that functions to combine the non-homogeneous physical devices contained in a data storage subsystem to create a logical device with new and unique quality of service characteristics that satisfy the criteria for the policies appropriate for the present data object. In particular, if there is presently no logical device that is appropriate for use in storing the present data object, the intelligent data storage manager defines a new logical device using existing physical and/or logical device definitions as component building blocks to provide the appropriate characteristics to satisfy the policy requirements. The intelligent data storage manager uses weighted values that are assigned to each of the presently defined logical devices to produce a best fit solution to the requested policies in an n-dimensional best fit matching algorithm. The resulting logical device definition is then implemented by dynamically interconnecting the logical devices that were used as the components of the newly defined logical device to store the data object.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 illustrates in block diagram form the overall architecture of a data storage subsystem in which the present intelligent data storage manager is implemented;

FIG. 2 illustrates a three-dimensional chart of the operating environment of the present intelligent data storage manager;

FIG. 3 illustrates one example of a virtual device that can be configured by the present intelligent data storage manager; and

FIG. 4 illustrates a three-dimensional chart of a user policy that must resolve priorities between two attributes: Cost per MB, and Time to First Byte.

DETAILED DESCRIPTION

FIG. 1 illustrates in block diagram form the overall architecture of a data storage subsystem 100 in which the present intelligent data storage manager 110 is implemented. The data storage subsystem is connected to a plurality of host processors 111-114 by means of a number of standard data channels 121-124. The data channels 121-124 are terminated in a host interface 101 which provides a layer of name servers 131-134 to present virtual implementations of existing defined physical device interfaces to the host processors 111-114. As far as the host processors 111-114 are concerned, the name servers 131-134 implement a real physical device. The name servers 131-134 convert the user data received from the host processor 111-114 into a user data object which can be either converted into a canonical format or left in binary format. The object handle server maps the object handle to logical device addresses and allows multiple instances of a data object. The object handle server 102 maps the user data object into a data space for storage. The mapping is determined by the policies programmed into the policy manager 105 of the data storage subsystem 100 and subject to security layer 103. The persistent storage for the object space is determined by the logical device manager 104 which allocates or creates a logical device based upon policies for storing the user data object. A logical device is a composite device and can consist of a real physical device such as a tape 151, a disk 152, optical disk 153, another logical device, such as Logical Device 1 which comprises a RAID 5 disk array 154, Logical Device N which comprises middleware software 155 that accesses another logical device, such as access of a logical device over a network connection, or combinations of the above. The logical device definition abstracts the nature of the real device associated with the persistent storage. The changes implemented in the technology of the persistent storage are thereby rendered transparent to the host application.

If there is presently no logical device that satisfies the criteria for the policies appropriate for a user data object, the logical device manager 104 creates a new logical device definition with the appropriate data storage characteristics to satisfy the policy requirements using existing physical and/or logical device definitions as component building blocks. The logical device manager 104 uses weighted values that are assigned to each of the presently defined logical devices to produce a best fit solution to the requested policies in an n-dimensional best fit matching algorithm. Thus, the intelligent data storage manager 110 maps the virtual device to the user data object rather than mapping a data object to a predefined data storage device. The various data storage attributes that are used by the intelligent data storage manager 110 to evaluate the appropriateness of a particular virtual device include, but are not limited to: speed of access to first byte, level of reliability, cost of storage, probability of recall, and expected data transfer rate. The logical device manager 104 stores the mapping data which comprises a real time definition of the available storage space in the data storage subsystem 100. Once one of the current logical device definitions meet the criteria required by a data object, the logical device manager 104 either allocates space on an existing instance of a logical device of that type or creates a new instance of that type of logical device.

POLICY ATTRIBUTES

The policy attributes and the potential algorithms that are used to map user requirements to storage devices are managed by the intelligent storage manager 110. A typical general set of attributes for storage devices is shown in Table 1:

TABLE-US-00001 TABLE 1 Policy Attributes Name of Attribute Range of Values (Dimension) Cost per MB (lg) $0.0001 to $1000.00 Time to first byte (lg) Ns to days Random read 0.0001 to 1000 MB/sec Random write 0.0001 to 1000 MB/sec Sequential read 0.0001 to 1000 MB/sec Sequential write 0.0001 to 1000 MB/sec Sequential (tape) or random 0 to 10 (where: 0 = sequential, 10 = (disk) storage or recall random) Size (lg) Bytes to petabytes Probability of recall 0 to 10 (where: 0 = lowest, 10 = highest) Virtual or real Device yes/no Level of reliability 0 to 10 (where: 0 = minimum, 10 = 100%) Others to be defined. . .

Each of these attributes has a range or dimension of "values". Each dimension needs to be relatively uniform in its number scheme. For example, each dimension could have a numeric value for 0.0 to 10.0. Some dimensions need to be logarithmic (lg) because of the inherent nature of the dimension. For example, Cost per MB can be defined as a logarithmic dimension that runs from the $0.001 for tape storage to $10 for RAM. So one approach is to do a distance calculation of the difference between the customer's policy requirements and each storage device's policy attributes. In addition, levels of priority among attributes can be specified since certain dimensions may be more important than others (reliability, for example). When the intelligent storage manager 110 must resolve between conflicting priority levels, the logical storage manager 104 tries to find ways to combine single devices into an optimal, logical device using logical combining operators.

Operation of the Intelligent Data Storage Manager

The present intelligent data storage manager 110 is responsive to one of the host processors 111 initiating a data write operation by transmitting a predefined set of commands over a selected one of the communication links to the data storage subsystem 100. These commands include a definition of the desired device on which the present data object is to be stored, typically in terms of a set of data storage characteristics. FIG. 2 illustrates a three-dimensional (of the above-noted multiple dimensions) chart of the operating environment of the present intelligent data storage manager 110 and the location of the host specified data storage device with respect to this environment. In particular, as mapped in a Cartesian coordinate system, the cost, data transfer rate, and data access time comprise the three axes used to measure the performance characteristics of the various physical 151-153 and virtual 154-155 devices of the data storage subsystem 100. As shown in FIG. 3, the standard tape 151, disk 152, and optical 153 devices each have a set of defined characteristics that can be mapped to the three-dimensional space of FIG. 2. The user has requested that their data be stored on a device, whose data storage characteristics do not match the data storage characteristics of any of the devices presently defined in the data storage subsystem 100. The desired data storage characteristics are shown mapped as a locus in the three-dimensional space in FIG. 2. The intelligent data storage manager 110 must therefore map the existing set of physical devices that are contained in the data storage subsystem 100 to satisfy the desired set of data storage characteristics defined by the user. This problem comprises a three-dimensional best fit mapping process wherein the set of available physical and virtual devices are mapped to match or at least approximate the desired set of data storage characteristics. This is accomplished by creating a composite virtual device that implements the defined desired data storage characteristics. For example, assume that the user has requested a data storage device that has a 20 MB/sec read performance and the data storage subsystem 100 is equipped with 5 MB/sec tape drives as one of the types of physical devices. The intelligent data storage manager 110 can create a 20 MB/sec data storage device by configuring a Redundant Array of Inexpensive Tape drives (RAIT) to connect a plurality of the existing tape drives 151 in parallel to thereby achieve the desired data throughput.

Examples of Operation of the Intelligent Data Storage Manager

There are many instances of data file storage where the needs of the application and/or user do not correspond to the reality of the data storage characteristics of the various data storage elements 151-153 and virtual data storage elements 154-155 that are available in the data storage subsystem 100. For example, the application "video on demand" may require a high reliability data storage element and fast access to the initial portion of the file, yet not require fast access for the entirety of the file since the data is typically read out at a fairly slow data access rate. However, the required data transfer bandwidth may be large, since the amount of data to be processed is significant and having a slow speed access device as well as a narrow bandwidth would result in unacceptable performance. Furthermore, the cost of data storage is a concern due to the volume of data. The intelligent data storage manager 110 must therefore factor all of these data storage characteristics to determine a best fit data storage device or devices to serve these needs. In this example, the defined data storage characteristics may be partially satisfied by a Redundant Array of Inexpensive Tapes since the reliability of this data storage device is high as is the data bandwidth, yet the cost of implementation is relatively low, especially if the configuration is a RAIT-5 and the data access speed is moderate. In making a determination of the appropriate data storage device, the intelligent data storage manager 110 must review the criticality of the various data storage characteristics and the amount of variability acceptable for that data storage characteristic.

Defining Attribute Values

All devices support some form of quality of service, which can be described as attributes with certain fixed values. For example, they cost $xxx per megabyte of data or have nnn access speed. The intelligent storage manager 110 provides an algorithmic way to use these attributes to determine the perfect device, as specified by user policy. In some cases, the perfect device is a logical device that is constructed when the intelligent storage manager 110 rank orders the distance between 1) how the user would like to have data stored and 2) the storage devices that are available. This logical device can span both disk and tape subsystems and, therefore, blurs the distinction between disk and tape.

The diagram of FIG. 4 shows an example of a user policy that must resolve priorities between two attributes: Cost per MB, and Time to First Byte. To resolve this, the intelligent storage manager 110 could create a logical device that is the mixture of disk and tape that best conforms to the specific policies the user has requested. In this example, some data could be stored on disk for quick access and some data could be stored on tape for lower cost of storage. Or the intelligent storage manager 110 could create a policy that migrates a small file between disk and tape over time: after a week the file would be transferred to tape to lower storage cost.

Table 2 provides a more complex comparison of device attributes versus attributes defined through user policy. In this example, the set of attributes of the following storage subsystems: single disk, RAID, single tape drive, and RAIT are listed. The intelligent storage manager 110 determines an optimal storage solution by doing a distance calculation between 1) the set of attributes for each device and 2) the set of attributes for a file (defined through user policy).

For example, the calculation below denotes the vector for point P by [x1(P), x2(P), x3(P)]. Then the distance between points 1 and 2 is {square root over ([(x1-x2).sup.2+(y1-y2).sup.2+(z1-z2).sup.2])}{square root over ([(x1-x2).sup.2+(y1-y2).sup.2+(z1-z2).sup.2])}{square root over ([(x1-x2).sup.2+(y1-y2).sup.2+(z1-z2).sup.2])} Where x1 is the attribute value defined by user policy. x2 is the attribute value defined for the device.

This example is for three dimensions. To extend it to more dimensions, take the difference between corresponding components of the two vectors, square this difference, add this square to all the other squares, and