Inter-partition message passing method, system and program product for throughput measurement in a partitioned processing environment Number:7,089,558 from the United States Patent and Trademark Office (PTO) owispatent A partitioned processing system is disclosed wherein a workload calculation is implemented for at least one partition. CPU utilization and I/O throughput information is used in calculating resource control parameters which is indicative of resource usage for the partition. The system collects and analyzes the computer system capacity data in a partition which determines a sizing metric. The system includes a manager in the computer system which issues a command to obtain throughput information of a computer system first partition, and issues a command to obtain resource utilization information of the computer system first partition. The manager then calculates a resource control parameter using the information obtained. The system includes a monitor which indicates resource allocation responsive to the resource control.<H environment, partitioned, Inter, partition, 7089558.html, Patent, pto, 7089558

Title: Inter-partition message passing method, system and program product for throughput measurement in a partitioned processing environment

Abstract: A partitioned processing system is disclosed wherein a workload calculation is implemented for at least one partition. CPU utilization and I/O throughput information is used in calculating resource control parameters which is indicative of resource usage for the partition. The system collects and analyzes the computer system capacity data in a partition which determines a sizing metric. The system includes a manager in the computer system which issues a command to obtain throughput information of a computer system first partition, and issues a command to obtain resource utilization information of the computer system first partition. The manager then calculates a resource control parameter using the information obtained. The system includes a monitor which indicates resource allocation responsive to the resource control.

Patent Number: 7,089,558 Issued on 08/08/2006 to Baskey, et al.

Inventors: Baskey; Michael E. (Wappingers Falls, NY), DeGilio; Frank J. (Poughkeepsie, NY), Merenda; Ann C. (Pleasant Valley, NY), Morrison; Timothy I. (Tillson, NY), Rohrbach; Christian F. (Poughkeepsie, NY), Temple, III; Joseph L. (Hurley, NY)
Assignee: International Business Machines Corporation (Armonk, NY)

Appl. No.: 09/801,407

Filed: March 8, 2001

Current U.S. Class: 718/104 ; 709/215; 709/223; 709/224; 709/226; 711/5; 715/736; 718/105

Current International Class: G06F 9/50 (20060101); G06F 12/00 (20060101); G06F 15/16 (20060101); G06F 9/00 (20060101); G06F 9/46 (20060101)

Field of Search: 718/100-104 709/200-253 711/5 717/130

References Cited [Referenced By]

U.S. Patent Documents


5241594	August 1993	Kung
5269009	December 1993	Herzl et al.
5414851	May 1995	Brice, Jr.
5434975	July 1995	Allen
5442802	August 1995	Brent et al.
5586260	December 1996	Hu
5768503	June 1998	Olkin
5771383	June 1998	Magee et al.
5931938	August 1999	Drogichen et al.
6301615	October 2001	Kutcher
6301616	October 2001	Pal et al.
6314501	November 2001	Gulick et al.
6510496	January 2003	Tarui et al.
6542926	April 2003	Zalewski et al.
6789054	September 2004	Makhlouf
2002/0129127	September 2002	Romero et al.

Other References

"Coupling Facility Configuration Options A Positioning Paper", S/390 Division, Jul. 1998, GF22-5042-00, pp. 1-29. cited by other .
"S/390 Cluster Technology: Parallel Sysplex", by J. M. Nick et al, IBM Systems Journal, vol. 36, No. 2, 1997, pp. 172-201. cited by other .
MQSeries--"Queue Manager Clusters", Fourth Edition, Nov. 2000, SC34-5349-03. cited by other .
OS/390 "Open Systems Adapter Support Facility User's Guide For OSA-2", Seventh Edition, Jun. 1999, SC28-1855-06. cited by other .
Enterprise Systems Architecture /390, Principles of Operation, Seventh Edition, Jul. 1999, SA22-7201-06. cited by other.

Primary Examiner: Bullock, Jr.; Lewis A.
Assistant Examiner: Tang; Kenneth
Attorney, Agent or Firm: Campbell; John E. Gonzalez; Floyd A.

Claims

We claim:

1. A system for the collection and analysis of computer system capacity data in a partitioned computer system having a computer system first partition and a computer system second partition, the system comprising: a network; a partitioned computer system in communication with the network wherein the partitioned computer system includes instructions to execute a method comprising the steps of: a) an analysis application running in a computer system second partition obtaining throughput information of a computer system first partition by way of a shared portion of memory, the shared portion of memory programmably accessible to both partitions, the shared memory for transferring information between the computer system first partition and the computer system second partition; b) the analysis application obtaining resource utilization information of the computer system first partition by way of a shared portion of memory; c) the analysis application calculating a resource control parameter using the resource utilization information obtained and the throughput information obtained, the resource control parameter comprising curve fitted throughput information as a function of resource utilization, wherein the curve fitted throughput information is derived from the throughput information obtained; d) providing the resource control parameter to a user agent, the resource control parameter indicating real time resource performance; and e) the user agent displaying the resource control parameter at a terminal.

2. The system according to claim 1 wherein the resource utilization information comprises CPU utilization.

3. The system according to claim 1 comprising the further step of the user agent displaying at a terminal, the resource control parameter wherein the resource control parameter comprises the throughput information as a function of resource utilization.

4. The system according to claim 1 comprising the further step of the user agent displaying as a graph at a terminal the resource control parameter, the display comprising effective utilization versus resource utilization wherein effective utilization derived in the calculating step comprises change in throughput divided by change in resource utilization.

5. The system according to claim 4 comprising the further step of the user agent displaying at a terminal a mark, the mark indicating the utilization at which the effective utilization is half of its maximum.

6. The system according to claim 1 comprising the further step of the user agent using the resource control parameter to adjust resources allocated to the first partition.

7. The system according to claim 6 wherein the using step is performed by a workload manager.

8. The system according to claim 7 wherein the workload manager is in a third partition.

9. The system according to claim 6 wherein the workload is managed by modifying resources allocated to the first partition.

10. The system according to claim 9 wherein the resources include I/O.

11. The system according to claim 9 wherein the resources include memory.

12. The system according to claim 9 wherein the resources include processors.

13. The system according to claim 6 wherein the workload is managed dynamically.

14. The system according to claim 1 comprising the further step of providing the throughput information and the resource utilization information for the calculating step using a single operation memory to memory transfer function.

15. The system according to claim 1 wherein the throughput information comprises network packet counts.

16. The system according to claim 1 wherein the throughput information comprises an inverse throughput.

17. The system according to claim 1 wherein the throughput information obtained comprises a network packet activity comprising a count of packets in and out of the system during an interval of time, wherein further the resource utilization obtained comprises CPU utilization during the interval of time, wherein the resource control parameter calculated comprises a representation of a relationship between the throughput information obtained and the resource utilization obtained.

18. The system according to claim 17 comprising the further step of providing any one of the throughput information or the resource utilization information for the calculating step by way of a shared portion of memory, the shared portion of memory programmably accessible to both partitions, the shared memory for transferring information between the computer system first partition and the computer system second partition.

19. A computer program product for the collection and analysis of computer system capacity data in a partitioned computer system having a computer system first partition and a computer system second partition, the computer program product comprising: a storage medium readable by a processing circuit and storing instructions for execution by a processing circuit for performing a method comprising the steps of: a) an analysis application running in a computer system second partition obtaining throughput information of a computer system first partition by way of a shared portion of memory, the shared portion of memory programmably accessible to both partitions, the shared memory for transferring information between the computer system first partition and the computer system second partition; b) the analysis application obtaining resource utilization information of the computer system first partition by way of a shared portion of memory; c) the analysis application calculating a resource control parameter using the resource utilization information obtained and the throughput information obtained, the resource control parameter comprising curve fitted throughput information as a function of resource utilization, wherein the curve fitted throughput information is derived from the throughput information obtained; d) providing the resource control parameter to a user agent, the resource control parameter indicating real time resource performance; and e) the user agent displaying the resource control parameter at a terminal.

20. The computer program product according to claim 19 wherein the resource utilization information comprises CPU utilization.

21. The computer program product according to claim 19 comprising the further step of the user agent displaying at a terminal, the resource control parameter wherein the resource control parameter comprises the throughput information as a function of resource utilization.

22. The computer program product according to claim 19 comprising the further step of the user agent displaying as a graph at a terminal the resource control parameter, the display comprising effective utilization versus resource utilization wherein effective utilization derived in the calculating step comprises change in throughput divided by change in resource utilization.

23. The computer program product according to claim 22 comprising the further step of the user agent displaying at a terminal a mark, the mark indicating the utilization at which the effective utilization is half of its maximum.

24. The computer program product according to claim 19 comprising the further step of the user agent using the resource control parameter to adjust resources allocated to the first partition.

25. The computer program product according to claim 24 wherein the using step is performed by a workload manager.

26. The computer program product according to claim 25 wherein the workload manager is in a third partition.

27. The computer program product according to claim 24 wherein the workload is managed by modifying resources allocated to the first partition.

28. The computer program product according to claim 27 wherein the resources include I/O.

29. The computer program product according to claim 27 wherein the resources include memory.

30. The computer program product according to claim 27 wherein the resources include processors.

31. The computer program product according to claim 24 wherein the workload is managed dynamically.

32. The computer program product according to claim 19 comprising the further step of providing the throughput information and the resource utilization information for the calculating step using a single operation memory to memory transfer function.

33. The computer program product according to claim 19 wherein the throughput information comprises network packet counts.

34. The computer program product according to claim 19 wherein the throughput information comprises an inverse throughput.

35. The computer program product according to claim 19 wherein the throughput information obtained comprises a network packet activity comprising a count of packets in and out of the system during an interval of time, wherein further the resource utilization obtained comprises CPU utilization during the interval of time, wherein the resource control parameter calculated comprises a representation of a relationship between the throughput information obtained and the resource utilization obtained.

36. The computer program product according to claim 35 comprising the further step of providing any one of the throughput information or the resource utilization information for the calculating step by way of a shared portion of memory, the shared portion of memory programmably accessible to both partitions, the shared memory for transferring information between the computer system first partition and the computer system second partition.

37. A computer implemented method for the collection and analysis of computer system capacity data in a partitioned computer system having a computer system first partition and a computer system second partition, the method comprising the steps of: a) an analysis application running in a computer system second partition obtaining throughput information of a computer system first partition by way of a shared portion of memory, the shared portion of memory programmably accessible to both partitions, the shared memory for transferring information between the computer system first partition and the computer system second partition; b) the analysis application obtaining resource utilization information of the computer system first partition by way of a shared portion of memory; c) the analysis application calculating a resource control parameter using the resource utilization information obtained and the throughput information obtained, the resource control parameter comprising curve fitted throughput information as a function of resource utilization, wherein the curve fitted throughput information is derived from the throughput information obtained; d) providing the resource control parameter to a user agent, the resource control parameter indicating real time resource performance; and e) the user agent displaying the resource control parameter at a terminal.

38. The method according to claim 37 wherein the resource utilization information comprises CPU utilization.

39. The method according to claim 38 comprising the further step of providing the throughput information and the resource utilization information for the calculating step using a single operation memory to memory transfer function.

40. The method according to claim 37 comprising the further step of the user agent displaying at a terminal, the resource control parameter wherein the resource control parameter comprises the throughput information as a function of resource utilization.

41. The method according to claim 37 comprising the further step of the user agent displaying as a graph at a terminal the resource control parameter, the display comprising effective utilization versus resource utilization wherein effective utilization derived in the calculating step comprises change in throughput divided by change in resource utilization.

42. The method according to claim 41 comprising the further step of the user agent displaying at a terminal a mark, the mark indicating the utilization at which the effective utilization is half of its maximum.

43. The method according to claim 37 comprising the further step of the user agent using the resource control parameter to adjust resources allocated to the first partition.

44. The method according to claim 43 wherein the using step is performed by a workload manager.

45. The method according to claim 44 wherein the workload manager is in a third partition.

46. The method according to claim 43 wherein the workload is managed by modifying resources allocated to the first partition.

47. The method according to claim 46 wherein the resources include I/O.

48. The method according to claim 46 wherein the resources include memory.

49. A method according to claim 46 wherein the resources include processors.

50. A method according to claim 43 wherein the workload is managed dynamically.

51. A method according to claim 37 wherein the throughput information comprises network packet counts.

52. A method according to claim 37 wherein the throughput information comprises an inverse throughput.

53. The method according to claim 37 wherein the throughput information obtained comprises a network packet activity comprising a count of packets in and out of the system during an interval of time, wherein further the resource utilization obtained comprises CPU utilization during the interval of time, wherein the resource control parameter calculated comprises a relationship between the throughput information obtained and the resource utilization obtained.

54. The method according to claim 53 comprising the further step of providing any one of the throughput information or the resource utilization information for the calculating step by way of a shared portion of memory, the shared portion of memory programmably accessible to both partitions, the shared memory for transferring information between the computer system first partition and the computer system second partition.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related, and cross-reference may be made to the following co-pending U.S. patent applications filed on even date herewith, each assigned to the assignee hereof, and each incorporated herein by reference: U.S. patent Ser. No. 09/801,993 to Kubala et al. for INTER-PARTITION MESSAGE PASSING METHOD, SYSTEM AND PROGRAM PRODUCT FOR MANAGING WORKLOAD IN A PARTITIONED PROCESSING ENVIRONMENT; U.S. patent Ser. No. 09/802,185 to Baskey et al. for INTER-PARTITION MESSAGE PASSING METHOD, SYSTEM AND PROGRAM PRODUCT FOR A SHARED I/O DRIVER; and U.S. patent Ser. No. 09/801,492 to Baskey et al. for INTER-PARTITION MESSAGE PASSING METHOD, SYSTEM AND PROGRAM PRODUCT FOR A SECURITY SERVER IN A PARTITIONED PROCESSING ENVIRONMENT.

FIELD OF THE INVENTION

This invention relates in general to partitioned data processing systems and in particular to uni-processor and multiprocessor systems capable of running multiple operating system images in the system's partitions, wherein each of the multiple operating systems may be an image of the same operating system in a homogeneous partitioned processing environment or wherein a plurality of operating systems are supported by the multiple operating system images in a heterogeneous partitioned processing environment.

BACKGROUND OF THE INVENTION

Most modern medium to large enterprises have evolved their IT infrastructure to extend the reach of their once centralized "glass house" data center throughout, and in fact beyond the bounds of their organization. The impetus for such evolution is rooted, in part, in the desire to interconnect heretofore disparate departmental operations, to communicate with suppliers and customers on a real-time basis, and is fueled by the burgeoning growth of the Internet as a medium for electronic commerce and the concomitant access to interconnection and business-to-business solutions that are increasingly being made available to provide such connectivity.

Attendant to this recent evolution is the need for modern enterprises to dynamically link many different operating platforms to create a seamless interconnected system. Enterprises are often characterized by a heterogeneous information systems infrastructure owing to such factors as non-centralized purchasing operations, application-based requirements and the creation of disparate technology platforms arising from merger related activities. Moreover, the desire to facilitate real-time extra-enterprise connectivity between suppliers, partners and customers presents a further compelling incentive for providing connectivity in a heterogeneous environment.

In response to a rapidly growing set of customer requirements, information technology providers have begun to devise data processing solutions that address these needs for extended connectivity for the enterprise data center.

Background information related to subject matter in this specification includes: U.S. patent Ser. No. 09/183,961 "COMPUTATIONAL WORKLOAD-BASED HARDWARE SIZER METHOD, SYSTEM AND PROGRAM PRODUCT" Ruffin et al. which describes analyzing the activity of a computer system; U.S. patent Ser. No. 09/584,276 "INTER-PARTITION SHARED MEMORY METHOD, SYSTEM AND PROGRAM PRODUCT FOR A PARTITIONED PROCESSING ENVIRONMENT" Temple et al. which describes shared memory between logical partitions; U.S. patent Ser. No. 09/253,246 "A METHOD OF PROVIDING DIRECT DATA PROCESSING ACCESS USING QUEUED DIRECT INPUT-OUTPUT DEVICE" Baskey et al which describes high bandwidth integrated adapters; U.S. patent Ser. No. 09/583,501 "Heterogeneous Client Server Method, System and Program Product For A Partitioned Processing Environment" Temple et al. which describes partitioning two different client servers in a system; IBM document SG24-5326-00 "OS/390 Workload Manager Implementation and Exploitation" ISBN: 0738413070 which describes managing workload of multiple partitions; and IBM document SA22-7201-06 ESA/390 Principles of Operation which describes the ESA/390 Instruction set architecture. These documents are incorporated herein by reference.

Initially, the need to supply an integrated system which simultaneously provides processing support for various applications which may have operational interdependencies, has led to an expansion in the market for partitioned multiprocessing systems. Once the sole province of the mainframe computer (such as the IBM S/390 system), these partitioned systems, which provide the capability to support multiple operating system images within a single physical computing system, have become available from a broadening spectrum of suppliers. For example, Sun Microsystems, Inc. has recently begun offering a form of system partitioning in the Ultra Enterprise 10000 high-end server which is described in detail in U.S. Pat. No. 5,931,938 to Drogichen et al. for "Multiprocessor Computer Having Configurable Hardware System Domains" filed Dec. 12, 1996 issued Aug. 3, 1999 and assigned to Sun Microsystems, Inc. Other companies have issued statements of direction indicating their interest in this type of system as well.

This industry adoption underscores the "systems within a system" benefits of system partitioning in consolidating various computational workloads within an enterprise onto one (or a few) physical server computers, and for simultaneously implementing test and production level codes in a dynamically reconfigurable hardware environment. Moreover, in certain partitioned multiprocessing systems such as the IBM S/390 computer system as described in the aforementioned cross-referenced patent applications, resources (including processors, memory and I/O) may be dynamically allocated within and between logical partitions depending upon the priorities assigned to the workload(s) being performed therein (IBM and S/390 are registered trademarks of International Business Machines Corporation). This ability to enable dynamic resource allocation based on workload priorities addresses long-standing capacity planning problems which have historically led data center managers to intentionally designate an excessive amount resources to their anticipated computational workloads to manage transient workload spikes.

While these partitioned systems facilitate the extension of the data center to include disparate systems throughout the enterprise, currently these solutions do not offer a straightforward mechanism for functionally integrating heterogeneous or homogeneous partitioned platforms into a single inter operating partitioned system. In fact, while these new servers enable consolidation of operating system images within a single physical hardware platform, they have not adequately addressed the need for inter-operability among the operating systems residing within the partitions of the server. This inter-operability concern is further exacerbated in heterogeneous systems having disparate operating systems in their various partitions. Additionally, these systems typically have not addressed the type of inter-partition resource sharing between such heterogeneous platforms which would enable a high-bandwidth, low-latency interconnection between the partitions. It is important to address these inter-operability issues since a system incorporating solutions to such issues would enable a more robust facility for communications between processes running in distinct partitions so as to leverage the fact that while such application are running on separate operating system, they are, in fact, local with respect to one another.

In the aforementioned U.S. patent Ser. No. 09/584,276 "INTER-PARTITION SHARED MEMORY METHOD, SYSTEM AND PROGRAM PRODUCT FOR A PARTITIONED PROCESSING ENVIRONMENT" by Temple et al., extensions to the "kernels" of the several operating systems facilitate the use of shared storage to implement cross partition memory sharing. A "kernel" is the core system services code in an operating system. While network message passage protocols can be implemented on the interface thus created, it is often desirable to enable efficient inter process communication without resorting to modification of one or more of the operating systems. It is also often desirable to avoid limiting the isolation of partitions in order to share memory regions as in aforementioned U.S. patent Ser. No. 09/584,276 by Temple et al. or as in the Sun Microsystems Ultra Enterprise 10000 high end server, as described in U.S. Pat. No. 5,931,938. At the same time it is desirable to pass information between partitions at memory speed instead of network speed. Thus a way to move memory between partition memories without sharing addresses is desired.

The IBM S/390 Gbit Ethernet (Asynchronous Coprocessor Data Mover Method and Means, U.S. Pat. No. 5,442,802, issued Aug. 15, 1995 and assigned to IBM) I/O adapter can be used to move data from one partition's kernel memory to another, but the data is moved from the first kernel memory to a queue buffer on the adapter and then transferred to a second queue buffer on the adapter before being transferred to a second kernel memory. This means that there is a total of three data movements in the transfer from memory to memory. In any message passing communications scheme, it is desirable to minimize the number of data movement operations so that the latency of data access approaches that of a single store and fetch to and from a shared storage. A move function has three data move operations for each block of data transferred. A way to remove one or two of these operations is desired.

Similarly, the IBM S/390 Parallel Sysplex Coupling Facility machine can and is used to facilitate inter partition message passing. However, in this case the transfer of data is from a first Kernel Memory to the coupling facility and then from the coupling facility to a second Kernel Memory. This requires two data operations rather than the single movement desired.

In many computer systems it is desirable to validate the identity of a user so that improper use of the data and applications on the machine through unauthorized or unwarranted access is prevented. Various operating and application systems have user authentication and other security services for this purpose. It is desirable to have users entering the partitioned system or indeed any cluster or network of systems to be validated only once on entry or at critical checkpoints such as request for critical resources, or execution of critical system maintenance functions. This desire is known as the "Single Sign on" requirement. Because of this the security servers of the various partitions must interact or be consolidated. Examples of this are the enhancement of the OS/390 SAF (RACF) interface to handle "digital certificates" received from the web, mapping them to the traditional user ID and password validation and entitlement within OS/390, Kerberos security servers, and the emerging LDAP standard for directory services.

Furthermore, because of the competitive nature of e-Commerce the performance of user authentication and entitlement is more important than in traditional systems. While a worker may expect to wait to be authenticated at the start of the day, a customer may simply go elsewhere if authentication takes too long. The use of encryption, because of the public nature of the web, exacerbates this problem. It is also often the case, that a device driver exists in one operating system that has not been written for others. In such cases it is desirable to interface to the device driver in one partition from another partition in an efficient manner. Only network connections are available for this type of operation today.

One of the problems with distributed systems is the management of "white space" or under utilized resources in one system, while other systems are over utilized. There are workload balancers such as IBM's LoadLeveler or Parallel Sysplex features of the OS/390 operating system workload manager which move work between systems or system images. It is possible and desirable in a partitioned computing system to shift resources rather than work between partitions. This is desirable because it avoids the massive context switching and data movement that comes with function shifting.

The "Sysplex Sockets" for IBM S/390 which uses the external clustering connections of the Sysplex to implement a UNIX operating system socket-to-socket connection is an example of some of the prior art. There, a service indicates the level of security available and sets up the connection based on the application's indication of security level required. However, in that case, encryption is provided for higher levels of security, and the Sysplex connection itself has a physical transport layer which was much deeper than the memory connections implemented by the present invention.

Similarly, a web server providing SSL authentication and providing certificate information (as a proxy) to a web application server can be seen as another example where sharing memory or direct memory to memory messages of the present invention are used to advantage. Here the proxy does not have to re-encrypt the data to be passed to the security server, and furthermore does not have a deep connection interface to manage. In fact it will be seen by those skilled in the art that in this embodiment of our invention the proxy server essentially communicates with the security server through a process which is essentially the same as a proxy server running under the same operating system as the security server. U.S. patent Ser. No. 09/411,417 "Methods, Systems and Computer Program Products for Enhanced Security Identity Utilizing an SSL Proxy" Baskey et al. discusses the use of proxy server to perform the secure sockets layer (SSL) in the secure HTTP protocol.

SUMMARY OF THE INVENTION

The foregoing problems and shortcomings of the prior art are addressed and overcome and further advantageous features are provided by the present invention which includes a partitioned computer system capable of supporting multiple heterogeneous operating system images wherein these operating system images may concurrently pass messages between their memory locations at memory speed without sharing memory locations. This is done by using an I/O adapter with a special device driver which together facilitate the movement of data from one kernel memory space of one partition directly to the kernel memory space of second partition.

One embodiment of the invention includes a system which collects and analyzes the computer system capacity data in a partition which determines a sizing metric. The system includes a manager in the computer system which issues a command to obtain throughput information of a computer system first partition, and issues a command to obtain resource utilization information of the computer system first partition. The manager then calculates a resource control parameter using the information obtained. The system includes a monitor which indicates resource allocation responsive to the resource control.

In an embodiment of the invention, the shared memory resource is independently mapped to the designated memory resource for plural inter operating processes running in the multiple partitions. In this manner, the common shared memory space is mapped by the process in each of the partitions sharing the memory resource to appear as memory resource assigned within the partition to that process and available for reading an writing data during the normal course of process execution.

In a further embodiment, the processes are interdependent and the shared memory resource may store from either or both processes for subsequent access by either or both processes.

In yet a further embodiment of the invention, the system includes a protocol for connecting the various processes within the partitions to the shared memory space.

In a another embodiment of the invention, the direct movement of data from a partition's kernel space to another partition's kernel space is enabled by an I/O adapter, which has physical access to all physical memory regardless of the partitioning. The ability of an I/O adapter to access all of memory is a natural consequence of the functions in a partitioned computer system which enables I/O resource sharing among the partitions. Such sharing is described in U.S. Pat. No. 5,414,851 issued May 9, 1995 for METHOD AND MEANS FOR SHARING I/O RESOURCES BY A PLURALITY OF OPERATING SYSTEMS, incorporated herein by reference. However the new and inventive adapter has the ability to move data from directly from one partition's memory to another partition's memory using a data mover.

In a further embodiment of the invention, the facilities for movement of data between kernel memories are implemented within the hardware and device driver of a network communication adapter.

In yet a further embodiment of the invention the network adapter is driven from a TCP/IP stack in each which is optimized for a local but heterogeneous secure connection through the memory to memory interface.

In another embodiment of the invention the data mover itself is implemented in the communication fabric of the partitioned processing system and controlled by the I/O adapter facilitating an even more direct memory to memory transfer.

In yet another embodiment of the invention, the data mover is controlled by the microcode of a privileged CISC instruction which can translate network addresses and offsets supplied as operands into physical addresses, whereby it performs the equivalent to a move character long instruction (IBM S/390 MVCL instruction, see IBM Document SA22-7201-06 "ESA/390 Principles of Operation") between physical addresses which have real and virtual addresses in two partitions.

In yet another embodiment of the invention, the data mover is controlled by a routine running in the hypervisor which has virtual and real memory access to all of physical memory and which can translate network addresses and offsets supplied as operands into physical addresses, whereby it performs the equivalent to a move character long instruction (IBM S/390 MVCL) between addresses which have real and virtual addresses in two partitions.

By implementing a server process in one of the partitions and client processes in other partitions, the partitioned system is capable of implementing a heterogeneous single system client server network. Since existing client/server processes typically inter-operate by network protocol connections they are easily implemented on message passing embodiments of the present invention gaining performance and security advantages without resorting to interface changes. However, implementation of client/server processes on the shared memory embodiments of the present invention can be advantageous in either performance or speed of deployment or both.

In a further embodiment of the present invention, the trusted/protected server environment is offered for application servers utilizing the shared memory or memory-to-memory message passing. This avoids the security exposure of externalizing authorization and authentication data without requiring additional encryption or authorization as in the current art.

In a specific embodiment of the present invention the Web server is the Linux Apache running under Linux for OS/390 communicating though a memory interface to a "SAF" security interface running under OS/390, Z/OS or VM/390. In this embodiment the Linux "Pluggable Authentication Module" is modified to drive the SAF interface through the memory connection.

In a further embodiment of the present invention a security server like Policy Director or RACF is modified so that the security credentials/context is stored in the shared memory or replicated via memory to memory transfers.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter which is regarded as constituting the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:

FIG. 1 illustrates a general overview of a partitioned data processing system;

FIG. 2 depicts a physically partitioned processing system having partitions comprised or one or more system boards;

FIG. 3 illustrates a logically partitioned processing system wherein the logically partitioned resources are dedicated to their respective partitions;

FIG. 4 illustrates a logically partitioned processing system wherein the logically partitioned resource may be dynamically shared between a number of partitions;

FIG. 5 illustrates the structure of UNIX operating system "Inter Process Communications";

FIG. 6 depicts an embodiment of the invention wherein real memory is shared according to a configuration table which is loaded by a stand alone utility;

FIG. 7A illustrates an embodiment of the present invention wherein the facilities of an I/O adapter and it's driver are used to facilitate the transfer of data among partitions;

FIG. 7B illustrates a prior art system of the embodiment of FIG. 7A;

FIG. 8 illustrates an embodiment of the present invention in which the actual data transfer between partitions is accomplished by a data mover implemented in the communication fabric of the partitioned data processing system;

FIG. 9 depicts components of an example data mover;

FIG. 10 shows an example format of a IBM S/390 move instruction;

FIG. 11 shows example steps of performing an Adapter Data Move;

FIG. 12 shows example steps of performing a processor data move;

FIG. 13 is a high level view of a Workload Manager (WLM);

FIG. 14 illustrates typical Workload Management Data;

FIG. 15 depicts clustering of client/server using indirect I/O; and

FIG. 16 depicts server clustering of client/server.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Before discussing the particular aspects of a preferred embodiment of the present invention, it will be instructive to review the basic components of a partitioned processing system. Using this as a backdrop will afford a greater understanding as to how the present inventions particular advantageous features may be employed in a partitioned system to improve the performance thereof. Reference should be made to IBM Document SC28-1855-06 "OS/390 V2R7.0 OSA/SF User's Guide" This book describes how to use the Open Systems Adapter Support Facility (OSA/AF), which is an element of the OS/390 operating system. It provides instructions for setting up OSA/SF and using either an OS/2 interface or OSA/SF commands to customize and manage OSAS. G321-5640-00 "S/390 cluster technology: Parallel Sysplex" describes a clustered multiprocessor system developed for the general-purpose, large-scale commercial marketplace. The S/390 Parallel Sysplex system is based on an architecture designed to combine the benefits of full data sharing and parallel processing in a highly scalable clustered computing environment. The Parallel Sysplex system offers significant advantages in the areas of cost, performance range, and availability. The IBM publication SC34-5349-01 "MQSeries Queue Manager Clusters" describes MQSeries queue manager clusters and explains the concepts, terminology and advantages of clusters. It summarizes the syntax of new and changed commands and shows a number of examples of tasks for setting up and maintaining clusters of queue managers. The IBM publication SA22-7201-06 "ESA/390 Principles of Operation" contains, for reference purposes, a detailed definition of the ESA/390 architecture. It is written as a reference for use primarily by assembler language programmers and describes each function at the level of detail needed to prepare an assembler language program that relies on that function; although anyone concerned with the functional details of ESA/390 will find it useful.

The aforementioned documents provide examples of the present state of the art and will be useful in understanding the background of the invention. These references are incorporated herein by reference.

Referring to FIG. 1, the basic elements constituting a partitioned processing system 100 is depicted. The system 100 is comprised of a memory resource block 101 which consists of a physical memory resource which is capable of being partitioned into blocks which are illustrated as blocks A and B, a processor resource block 102 which may consist of one or more processors which may be logically or physically partitioned to coincide with the partitioned memory resource 101, and an input/output (I/O) resource block 103 which may be likewise partitioned. These partitioned resource blocks are interconnected via an interconnection fabric 104 which may comprise a switching matrix, etc. It will be understood that the interconnection fabric 104 may serve the function of interconnecting resources within a partition, such as connecting processor 102B to memory 101B and may also serve to interconnect resources between partitions such as connecting processor 102A to memory 101B. The term "Fabric" used in this specification is intended to mean the generic methods known in the art for interconnecting elements of a system. It may be a simple point to point bus or a sophisticated routing mechanism. While the present set of figures depicts systems having two partitions (A and B) it will be readily appreciated that the such a representation has been chosen to simplify this description and further that the present invention is intended to encompass systems which may be configured to implement as many partitions as the available resources and partitioning technology will allow.

Upon examination, it will be readily understood that each of the illustrated partitions A and B taken separately comprise the constituent elements of a separate data processing system i.e., processors, memory and I/O. This fact is the characteristic that affords partitioned processing systems their unique "systems within a system" advantages. In fact, and as will be illustrated herein, the major distinction between currently available partitioned processing systems is the boundary along which the system resources may be partitioned and the ease with which resources may be moved across these boundaries between partitions.

The first case, where the boundary separating partitions is a physical boundary, is best exemplified by the Sun Microsystems Ultra Enterprise 10000 system. In the Ultra Enterprise 10000 system, the partitions are demarked along physical boundaries, specifically, a domain or partition consists of one or more physical system boards each of which comprises a number of processors, memory and I/O devices. A domain is defined as one or more of these system boards and the I/O adapters attached thereto. The domains are in turn interconnected by a proprietary bus and switch architecture.

FIG. 2 illustrates a high level representation of the elements constituting a physically partitioned processing system 200. As can be seen via reference to FIG. 2, the system 200 includes two domains or partitions A and B. Partition A is comprised of two system boards 201A1 and 201A1. Each system board of partition A includes memory 201A, processors 202A, I/O 203A and an interconnection medium 204A. Interconnection medium 204A allows the components on system board 201A1 to communicate with one another. Similarly, partition B, which is comprised of a single system board includes like constituent processing elements: memory 201B, processors 202B, I/O 203B and interconnect 204B. In addition to the system boards grouped into partitions, there exists an interconnection fabric 205 which is coupled to each of the system boards and permits interconnections between system boards within a partition as well as the interconnection of system boards in different partitions.

The next type of system partition is termed logical partitioning. In such systems there is no physical boundary constraining the assignment of resources to the various partitions, but rather the system may be viewed as having an available pool of resources, which, independent of their physical location, may be assigned to any of the partitions. This is a distinction between a physically partitioned system wherein, for example, all of the processors on a given system board (such as system board 201A1) are, of necessity, assigned to the same partition. The IBM AS/400 system exemplifies a logically partitioned dedicated resource processing system. In the AS/400 system, a user may include processors, memory and I/O in a given partition irrespective of their physical location. So, for example, two processors physically located on the same card may be designated as resources for two different partitions. Likewise, a memory resource in a given physical package such as a card may have a portion of its address space logically dedicated to one partition and the remainder dedicated to another partition.

A characteristic of logically partitioned dedicated resource systems, such as the AS/400 system, is that the logical mapping of a resource to a partition is a statically performed assignment which can only undergo change by manual reconfiguration of the system. Referring to FIG. 3, the processor 302A1 represents a processor that can be physically located anywhere in the system and which has been logically dedicated to partition A. If a user wishes to re-map processor 302A1 to partition B, the processor would have to be taken off-line and manually re-mapped to accommodate the change. The logically partitioned system provides a greater granularity for resource partitioning as it is not constrained by the limitation of a physical partitioning boundary such as the a system board which, for example, supports a fixed number of processors. However, reconfiguration of such a logically partitioned, dedicated resource system cannot be undertaken without disrupting the operation of the resource undergoing the partition remapping. It can therefore be seen, that while such a system avoids some of the limitations inherent in a physically partitioned system, it still has reconfiguration restraints associated with the static mapping of resources among partitions.

This brings us to the consideration of the logically partitioned, shared resource system. An example of such a system is the IBM S/390 computer system. A characteristic of logically partitioned, shared resource system is that a logically partitioned resource such as a processor may be shared by more than one partition. This feature effectively overcomes the reconfiguration restraints of the logically partitioned, dedicated resource system.

FIG. 4 depicts the general configuration of a logically partitioned, resource sharing system 400. Similar to the logically partitioned, dedicated resource system 300, system 400 includes memory 401, processor 402 and I/O resource 403 which may be logically assigned to any partition (A or B in our example) irrespective of its physical location in the system. As can be seen in system 400 however, the logical partition assignment of a particular processor 402 or I/O 403 may be dynamically changed by swapping virtual processors (406) and I/O drivers (407) according to a scheduler running in a "Hypervisor" (408). (A Hypervisor is a supervisory program that schedules and allocates resources for virtual machines). The virtualization of processors and I/O allows entire operating system images to be swapped in an out of operation with appropriate prioritization allowing partitions to share these resources dynamically.

While the logically partitioned, shared resource system 400 provides a mechanism for sharing processor and I/O resource, inter-partition message passing has not been fully addressed by existing systems. This is not to say that the existing partitioned system cannot enable communication among the partitions. In fact, such communication occurs in each type of partitioned system as described herein. However, none of these implementations provides a means to move data from kernel memory to kernel memory without the intervention of a hypervisor, a shared memory implementation, or a standard set of adapters or channel communication devices or network connecting the partitions.

In the physically partitioned multiprocessing systems typified by the Sun Microsystems Ultra Enterprise 10000 system, as described in U.S. Pat. No. 5,931,938, an area of system memory may be accessible by multiple partitions at the hardware level, by setting mask registers appropriately. The Sun patent does not teach how to exploit this capability other than to note that it can be used as a buffering mechanism and communication means for inter partition networks. Aforementioned U.S. patent Ser. No. 09/584,276, Temple et al. teaches how to build and exploit a shared memory mechanism in a heterogeneous partitioned system.

In the IBM S/390 system, as detailed in "Coupling Facility Configuration Options: A Positioning Paper" (GF22-5042-00, IBM Corp.) similar internal clustering capability is described for using commonly addressed physical memory as an "integrated coupling facility". Here the shared storage is indeed a repository, but the connection to it is through an I/O like device driver called XCF. Here the shared memory is implemented in the coupling facility, but requires non S/390 operating systems to create extensions to use it. Furthermore, this implementation causes data to be moved from the one partition's kernel memory to the coupling facility's memory and then to a second partition's kernel memory.

A kernel is the part of an operating system that performs basic functions such as allocating hardware resources. A kernel memory is the memory space available to a kernel for use by the kernel to execute it's function.

By contrast, the present invention provides a means for moving the data from one partition's kernel memory to another partition's kernel memory in one operation using the enabling facilities of a new I/O adapter and its device driver, without providing for shared storage extensions to the operating systems in either partition or in the hardware.

To understand how the present invention is realized, it is useful to understand inter process communications in an operating system. Referring to FIG. 5, Processes A (501) and B (503) each have address spaces Memory A (502) and Memory B (504). These addresses spaces have real memory allocated to them by the execution of system calls by the Kernel (505). The Kernel has its own address space, Memory K (506). In one form of communication, Process A and B communicate by the creation of a buffer 510 in Memory K, by making the appropriate system calls to create, connect to and access the buffer 510. The semantics of these calls vary from system to system, but the effect is the same. In a second form of communication a segment 511 of Memory S (507) is mapped into the address spaces of Memory A (502) and Memory B (504). Once this mapping is complete, then Processes A (501) and B (503) are free to use the shared segment of Memory S (507) according to any protocol which both processes understand.

U.S. patent Ser. No. 09/583,501 "Heterogeneous Client Server Method, System and Program Product For A Partitioned Processing Environment" is represented by FIG. 6 in which Processes A (601) and B (603) reside in different operating system domains, images, or partitions (Partition 1 (614) and Partition 2 (615)). There are now Kernel 1 (605) and Kernel 2 (607) which have Memory K1 (606) and Memory K2 (608) as their Kernel memories. Memory S (609) is now a space of physical memory accessible by both Partition 1 and Partition 2. The enablement of such sharing can be according to any implementation including without limitation the UE10000 memory mapping implementation or the S/390 hypervisor implementation, or any other means to limit the barrier to access which is created by partitioning. As an alternative example, the shared memory is mapped into the very highest physical memory addresses, with the lead ones in a configuration register defining the shared space.

By convention, Memory S (609) has a shared segment (610) which is used by extensions of Kernel 1 and Kernel 2 which is mapped into Memory K1 and Memory K2. Segment 610 is used to hold the definition and allocation tables for segments of Memory (609), which are mapped to Memory K1 (606) and Memory K2 (608) allowing cross partition communication according to the first form described above or to define a segment S2 (611) mapped into Memory A (602) and Memory B (604) according to the second form of communication described above with reference to FIG. 5. In an embodiment of the invention Memory S is of limited size and is pinned in real storage. However, it is contemplated that memory need not be pinned, enabling a larger share storage space, so long as the attendant page management tasks were efficiently managed.

In a first embodiment of the referenced invention the definition and allocation tables for the shared storage are set up in memory by a stand alone utility program called Shared Memory Configuration Program (SMCP) (612) which reads data from a Shared Memory Configuration Data Set (SMCDS) (613) and builds the table in segment S1 (610) of Memory S (609). Thus, the allocation and definition of which kernels share which segments of storage is fixed and predetermined by the configuration created by the utility. The various kernel extensions then use the shared storage to implement the various inter-image, inter-process communication constructs, such as pipes, message queues, sockets and even allocating some segments to user processes as shared memory segments according to their own conventions and rules. These inter-process communications are enable through IPC APIs 618 and 619.

The allocation table for the shared storage contains entries which consist of image identifiers, segment numbers, gid, uid, "sticky bit" and permission bits. A sticky bit indicates that the related store is not page-able. In this example embodiment, the sticky bit is reserved and in assumed to be 1 (IE, the data is pinned or "stuck" in memory at this location.). Each group, user, and image which uses a segment has an entry in the table. By convention all kernels can read the table but none can write it. At initialization the kernel extension reads the configuration table and creates its own allocation table for use when cross image inter process communication is requested by other processes. Some or all of the allocated space is used by the kernel for the implementation of "pipes", files and message queues which it creates at the request of other processes which request inter-process communications. A pipe is data from one process directed through a kernel function to a second process. Pipes, files and message queues are standard UNIX operating system inter process communication API's and data structures as used in Linux, OS/390 USS, and most UNIX operating systems. A portion of the shared space may be mapped by a further kernel extension into the address spaces of other processes for direct cross system memory sharing.

The allocation, use of, and mapping shared memory to virtual address spaces is done by each kernel according to its own conventions and translation processes, but the fundamental hardware locking and memory sharing protocols are driven by the common hardware design architecture which underlies the rest of the system.

The higher level protocols must be common in order for communication to occur. In the preferred embodiment this is done by having each of the various operating systems images implement the IPC (Inter Process Communications) API for use with the UNIX operating system, with the extension identifying the request as cross image. This extension can be by parameter or by separate new identifier/command name.

Referring to FIGS. 4 and 7A, one can see that the present invention avoids both the transfer of data over a channel or network connection and the use of a shared memory extension to the operating system. An application process (701) in partition 714 accesses socket interface 708 which calls kernel 1 (705). A socket interface is a construct that relates a specific port of the TCP/IP stack to a listening user process. The kernel accesses the device driver (716) which causes data to be transferred from kernel memory 1 (706) to kernel memory 2 (708), by and through the hardware of the I/O adapter (720) in what looks to the memory (401) like a memory to memory move, bypassing the cache memories implemented in the processors (402) and/or fabric (404) of partitions 714 and 715. Having moved the data I/O adapter then accesses the device driver (717) in partition 715, indicating that the data has been moved. The device driver 717 then