Title: Data storage system having separate data transfer section and message network with plural directions on a common printed circuit board
Abstract: A system interface having a plurality of first directors and a crossbar switch having input/output ports coupled to the first directors on such one of the first director boards and a pair of output/input ports. A plurality of second director boards is provided. Each one of the second directors boards has a plurality of second directors a crossbar switch having input/output ports coupled to the second directors on such one of the second director boards and a pair of output/input ports. A data transfer section is provided having a cache memory. The cache memory is coupled to the plurality of first and second directors. A message network is provided, such network being operative independently of the data transfer section. The first and second directors control data transfer between the first directors and the second directors in response to messages passing between the first directors and the second directors through the message network to facilitate data transfer between first directors and the second directors. The data passes through the cache memory in the data transfer section. Each one of the directors includes a data pipe coupled between an input of such one of the first directors and the cache memory; a microprocessor. A controller is coupled to the microprocessor and the data pipe for controlling the transfer of the messages between the message network and such one of the first directors and for controlling the data between the input of such one of the first directors and the cache memory.
Patent Number: 7,003,601 Issued on 02/21/2006 to MacArthur
| Inventors:
|
MacArthur; Stephen D. (Northborough, MA)
|
| Assignee:
|
EMC Corporation (Hopkinton, MA)
|
| Appl. No.:
|
539966 |
| Filed:
|
March 31, 2000 |
| Current U.S. Class: |
710/74; 710/2; 710/5; 710/62; 709/203; 709/213; 711/147 |
| Current Intern'l Class: |
G06F 13/12 (20060101) |
| Field of Search: |
711/111-114
710/29-38
|
References Cited [Referenced By]
U.S. Patent Documents
| 4438494 | Mar., 1984 | Budde et al.
| |
| 4456987 | Jun., 1984 | Wirsing.
| |
| 4476526 | Oct., 1984 | Dodd.
| |
| 4480307 | Oct., 1984 | Budde et al.
| |
| 4484262 | Nov., 1984 | Sullivan et al.
| |
| 4633387 | Dec., 1986 | Hartung.
| |
| 4636946 | Jan., 1987 | Hartung et al.
| |
| 4636948 | Jan., 1987 | Gdaniec et al.
| |
| 4688168 | Aug., 1987 | Gudatis et al.
| |
| 4780808 | Oct., 1988 | Moreno et al.
| |
| 4800483 | Jan., 1989 | Yamamoto et al.
| |
| 4977556 | Dec., 1990 | Noguchi.
| |
| 5115433 | May., 1992 | Baran et al.
| |
| 5119485 | Jun., 1992 | Ledbetter, Jr. et al.
| |
| 5206939 | Apr., 1993 | Yanai et al.
| |
| 5214768 | May., 1993 | Martin et al.
| |
| 5241640 | Aug., 1993 | Hisano et al.
| |
| 5263145 | Nov., 1993 | Brady et al.
| |
| 5269011 | Dec., 1993 | Yanai et al.
| |
| 5274645 | Dec., 1993 | Idleman et al.
| |
| 5335327 | Aug., 1994 | Hisano et al.
| |
| 5386511 | Jan., 1995 | Murata et al.
| |
| 5406607 | Apr., 1995 | Marietta.
| |
| 5448564 | Sep., 1995 | Thor.
| |
| 5459856 | Oct., 1995 | Inoue.
| |
| 5479611 | Dec., 1995 | Oyama.
| |
| 5539733 | Jul., 1996 | Anderson et al.
| |
| 5551048 | Aug., 1996 | Steely, Jr.
| |
| 5574865 | Nov., 1996 | Hashemi.
| |
| 5586264 | Dec., 1996 | Belknap et al.
| |
| 5603058 | Feb., 1997 | Belknap et al.
| |
| 5689728 | Nov., 1997 | Sugimoto et al.
| |
| 5703875 | Dec., 1997 | Burnett.
| |
| 5742789 | Apr., 1998 | Ofer et al.
| |
| 5745790 | Apr., 1998 | Oskouy.
| |
| 5752256 | May., 1998 | Fujii et al.
| |
| 5787265 | Jul., 1998 | Leshem.
| |
| 5805821 | Sep., 1998 | Saxena et al.
| |
| 5813024 | Sep., 1998 | Saito.
| |
| 5819054 | Oct., 1998 | Ninomiya et al.
| |
| 5819104 | Oct., 1998 | Tuccio.
| |
| 5839906 | Nov., 1998 | Leshem.
| |
| 5890207 | Mar., 1999 | Sne et al.
| |
| 5903911 | May., 1999 | Gaskins.
| |
| 5920893 | Jul., 1999 | Nakayama et al.
| |
| 5948062 | Sep., 1999 | Tzelnic et al.
| |
| 5949982 | Sep., 1999 | Frankeny et al.
| |
| 6009481 | Dec., 1999 | Mayer.
| |
| 6038638 | Mar., 2000 | Cadden et al.
| |
| 6038641 | Mar., 2000 | Zangenehpour.
| |
| 6055603 | Apr., 2000 | Ofer et al.
| |
| 6061274 | May., 2000 | Thibault et al.
| |
| 6081860 | Jun., 2000 | Bridges et al.
| |
| 6125429 | Sep., 2000 | Goodwin et al.
| |
| 6134624 | Oct., 2000 | Burns et al.
| |
| 6178466 | Jan., 2001 | Gilbertson et al.
| |
| 6205536 | Mar., 2001 | Yoshida.
| |
| 6230229 | May., 2001 | Van Krevelen et al.
| |
| 6240335 | May., 2001 | Wehrung et al.
| |
| 6275877 | Aug., 2001 | Duda.
| |
| 6275953 | Aug., 2001 | Vahalia et al.
| |
| 6304903 | Oct., 2001 | Ward.
| |
| 6317805 | Nov., 2001 | Chilton et al.
| |
| 6338095 | Jan., 2002 | Yasuda et al.
| |
| 6378029 | Apr., 2002 | Venkitakrishnan et al.
| |
| 6389494 | May., 2002 | Walton et al.
| |
| 6397281 | May., 2002 | MacLellan et al.
| |
| 6418496 | Jul., 2002 | Pawlowski et al.
| |
| 6438586 | Aug., 2002 | Hass et al.
| |
| 6470389 | Oct., 2002 | Chung et al.
| |
| 6529521 | Mar., 2003 | MacArthur.
| |
| 6557140 | Apr., 2003 | Kakuta et al.
| |
| 6563793 | May., 2003 | Golden et al.
| |
| 6651130 | Nov., 2003 | Thibault.
| |
| Foreign Patent Documents |
| 0510821 | Oct., 1992 | EP.
| |
| 0795812 | Sep., 1997 | EP.
| |
| 2 325 541 | Nov., 1998 | GB.
| |
| 2 360 377 | Sep., 2001 | GB.
| |
| 2366425 | Mar., 2002 | GB.
| |
| 7-302171 | Jul., 1989 | JP.
| |
| 6337836 | Dec., 1994 | JP.
| |
| 7191930 | Jul., 1995 | JP.
| |
| 07244634 | Sep., 1995 | JP.
| |
| 7-302171 | Nov., 1995 | JP.
| |
| 8172446 | Jul., 1996 | JP.
| |
| 93/01553 | Jan., 1993 | WO.
| |
| WO 99/2615/0 | May., 1999 | WO.
| |
Other References
F. Halsall, "The Role of DQDB", 1994, The Institution of Electrical Engineers,
IEE, Savory Place, London WC2R OBL.UK.
U.S. Appl. No. 09/540,825, filed Mar. 31, 2000.
U.S. Appl. No. 10/054,241, filed Jan. 22, 2002.
U.S. Appl. No. 09/540,828, filed Mar. 31, 2000.
U.S. Appl. No. 09/606,734, filed Jun. 29, 2000.
U.S. Appl. No. 09/606,730, filed Jun. 29, 2000.
U.S. Appl. No. 10/120,957, filed Apr. 11, 2002.
U.S. Appl. No. 09/680,156, filed Oct. 5, 2000.
|
Primary Examiner: Huynh; Kim
Assistant Examiner: Sorrell; Eron
Claims
What is claimed is:
1. A system interface comprising:
(a) a plurality of first director boards, each one of the first director boards having:
(i) a plurality of first directors; and
(ii) a crossbar switch having input/output ports coupled to the first directors
on such one of the first director boards and a pair of output/input ports;
(b) a plurality of second director boards, each one of the second directors boards having:
(i) a plurality of second directors; and
(ii) a crossbar switch having input/output ports coupled to the second directors
on such one of the second director boards and a pair of output/input ports;
(c) a data transfer section having a cache memory, such cache memory being coupled
to the plurality of first and second directors;
(d) a message network, operative independently of the data transfer section,
coupled to the pair of output/input ports of each one of the directors boards of
the plurality of first director boards and to the pair of output/input ports of
each one of the directors boards of the plurality of second director boards; and
(e) wherein the first and second directors control data transfer between the
first directors and the second directors in response to messages passing between
the first directors and the second directors through the message network to facilitate
data transfer between first directors and the second directors with such data passing
through the cache memory in the data transfer section.
2. The system interface recited in claim 1 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a controller for transferring the messages between the message network and such
one of the first directors.
3. The system interface recited in claim 1 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
4. The system interface recited in claim 2 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
5. The system interface recited in claim 1 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the first
directors and for controlling the data between the input of such one of the first
directors and the cache memory.
6. The system interface recited in claim 5 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
7. The system interface recited in claim 1 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
8. A data storage system for transferring data between a host computer/server
and a bank of disk drives through a system interface, such system interface comprising:
(a) a plurality of first director boards coupled to host computer/server; each
one of the first director boards having:
(i) a plurality of first directors; and
(ii) a crossbar switch having input/output ports coupled to the first directors
on such one of the first director boards and a pair of output/input ports;
(b) a plurality of second director boards coupled to the bank of disk drives,
each one of the second director boards having:
(i) a plurality of second directors; and
(ii) a crossbar switch having input/output ports coupled to the second directors
on such one of the second director boards and a pair of output/input ports;
(c) a data transfer section having a cache memory, such cache memory being coupled
to the plurality of first and second directors;
(d) a message network, operative independently of the data transfer section,
coupled to the pair of output/input ports of each one of the directors boards of
the plurality of first director boards and to the pair of output/input ports of
each one of the directors boards of the plurality of second director boards; and
(e) wherein the first and second directors control data transfer between the
host computer and the bank of disk drives in response to messages passing between
the first directors and the second directors through the message network to facilitate
the data transfer between host computer/server and the bank of disk drives with
such data passing through the cache memory in the data transfer section.
9. The system interface recited in claim 8 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a controller for transferring the messages between the message network and such
one of the first directors.
10. The system interface recited in claim 9 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
11. The system interface recited in claim 8 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
12. The system interface recited in claim 8 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the first
directors and for controlling the data between the input of such one of the first
directors and the cache memory.
13. The system interface recited in claim 12 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
14. The system interface recited in claim 8 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
15. A system interface comprising:
(a) a plurality of first director boards, each one of the first director boards having:
(i) a plurality of first directors; and
(ii) a crossbar switch having input/output ports coupled to the first directors
on such one of the first director boards and an output/input port;
(b) a plurality of second director boards, each one of the second directors boards having:
(i) a plurality of second directors; and
(ii) a crossbar switch having input/output ports coupled to the second directors
on such one of the second director boards and an output/input port;
(c) a data transfer section having a cache memory, such cache memory being coupled
to the plurality of first and second directors;
(d) wherein the data transfer section is also coupled to the output/input port
of the crossbar switch of each one of the plurality of first director boards and
to the output/input port of the crossbar switch of each one of the plurality of
second director boards;
(e) a message network, operative independently of the data transfer section; and
(f) wherein the first and second directors control data transfer between the
first directors and the second directors in response to messages passing between
the first directors and the second directors through the message network to facilitate
data transfer between first directors and the second directors with such data passing
through the cache memory in the data transfer section.
16. The system interface recited in claim 15 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a controller for transferring the messages between the message network and such
one of the first directors.
17. The system interface recited in claim 16 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
18. The system interface recited in claim 15 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
19. The system interface recited in claim 15 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the first
directors and for controlling the data between the input of such one of the first
directors and the cache memory.
20. The system interface recited in claim 19 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
21. The system interface recited in claim 15 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
22. A data storage system for transferring data between a host computer/server
and a bank of disk drives through a system interface, such system interface comprising:
(a) a plurality of first director boards coupled to host computer/server; each
one of the first director boards having:
(i) a plurality of first directors; and
(ii) a crossbar switch having input/output ports coupled to the first directors
on such one of the first director boards and an output/input port;
(b) a plurality of second director boards coupled to the bank of disk drives,
each one of the second director boards having:
(i) a plurality of second directors; and
(ii) a crossbar switch having input/output ports coupled to the second directors
on such one of the second director boards and an output/input port;
(c) a data transfer section having a cache memory, such cache memory being coupled
to the plurality of first and second directors;
(d) wherein the data transfer section is also coupled to the output/input port
of the crossbar switch of each one of the plurality of first director boards and
to the output/input port of the crossbar switch of each one of the plurality of
second director boards;
(e) a message network, operative independently of the data transfer section; and
(f) wherein the first and second directors control data transfer between the
host computer and the bank of disk drives in response to messages passing between
the first directors and the second directors through the message network to facilitate
the data transfer between host computer/server and the bank of disk drives with
such data passing through the cache memory in the data transfer section.
23. The system interface recited in claim 22 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a controller for transferring the messages between the message network and such
one of the first directors.
24. The system interface recited in claim 23 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
25. The system interface recited in claim 22 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a controller for transferring the messages between the message network and such
one of the second directors.
26. The system interface recited in claim 22 wherein each one of the first directors includes:
a data pipe coupled between an input of such one of the first directors and the
cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the first
directors and for controlling the data between the input of such one of the first
directors and the cache memory.
27. The system interface recited in claim 26 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
28. The system interface recited in claim 22 wherein each one of the second directors includes:
a data pipe coupled between an input of such one of the second directors and
the cache memory;
a microprocessor; and
a controller coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the second
directors and for controlling the data between the input of such one of the second
directors and the cache memory.
29. A system interface comprising:
(a) a plurality of first director boards, each one of the first director boards having:
(i) a plurality of first directors; and
(ii) a crossbar switch having input/output ports coupled to the first directors
on such one of the first director boards and an output/input port;
(b) a plurality of second director boards, each one of the second directors boards having:
(i) a plurality of second directors; and
(ii) a crossbar switch having input/output ports coupled to the second directors
on such one of the second director boards and an output/input port;
(c) a data transfer section having a cache memory, such cache memory being coupled
to the plurality of first and second directors;
(d) wherein the data transfer section is also coupled to the output/input port
of the crossbar switch of each one of the plurality of first director boards and
to the output/input port of the crossbar switch of each one of the plurality of
second director boards;
(e) a message network, operative independently of the data transfer section; and
(f) wherein the first and second directors control data transfer between the
first directors and the second directors in response to messages passing between
the first directors and the second directors through the message network with such
messages by-passing the data transfer section and with such data transfer comprising
passing data through the directors to the cache memory in the data transfer section.
30. A system interface comprising:
(a) a plurality of first director boards, each one of the first director boards having:
(i) a plurality of first directors, each one of the directors having a data port
and a message port; and
(ii) a crossbar switch having input/output ports coupled to the message ports
of the first directors on such one of the first director boards and a pair of output/input ports;
(b) a plurality of second director boards, each one of the second directors boards having:
(i) a plurality of second directors, each one of the directors having a data
port and a message port; and
(ii) a crossbar switch having input/output ports coupled to the message ports
of the second directors on such one of the second director boards and a pair of
output/input ports;
(c) a data transfer section having a cache memory, such cache memory being coupled
to the data ports of the plurality of first and second directors;
(d) a message network, coupled to the pair of output/input ports of each one
of the directors boards of the plurality of first director boards and to the pair
of output/input ports of each one of the directors boards of the plurality of second
director boards; and
(e) wherein the first and second directors control data transfer between first
director and the second director with data in such data transfer passing through
the cache memory in response to messages passing between the first director and
the second director through the message network.
31. A system interface comprising:
(a) a plurality of first director boards, each one of the first director boards having:
(i) a plurality of first directors; and
(ii) a crossbar switch having input/output ports coupled to the first directors
on such one of the first director boards and a pair of output/input ports;
(b) a plurality of second director boards, each one of the second directors boards having:
(i) a plurality of second directors; and
(ii) a crossbar switch having input/output ports coupled to the second directors
on such one of the second director boards and a pair of output/input ports;
(c) a data transfer section having a cache memory, such cache memory being coupled
to the plurality of first and second directors;
(d) a message network, operative independently of the data transfer section,
coupled to the pair of output/input ports of each one of the directors boards of
the plurality of first director boards and to the pair of output/input ports of
each one of the directors boards of the plurality of second director boards;
(e) wherein the first and second directors control data transfer between first
director and the second director with data in such data transfer passing through
the cache memory in response to messages passing between the first director and
the second director through the message network; and
(f) wherein each one of the messages includes a destination field.
32. A system interface comprising:
(a) a plurality of first director boards, each one of the first director boards having:
(i) a plurality of first directors; and
(ii) a crossbar switch having input/output ports coupled to the first directors
on such one of the first director boards and a pair of output/input ports;
(b) a plurality of second director boards, each one of the second directors boards having:
(i) a plurality of second directors; and
(ii) a crossbar switch having input/output ports coupled to the second directors
on such one of the second director boards and a pair of output/input ports;
(c) a data transfer section having a cache memory, such cache memory being accessible
to the plurality of first and second directors through arbitration;
(d) a message network, operative independently of the data transfer section,
coupled to the pair of output/input ports of each one of the directors boards of
the plurality of first director boards and to the pair of output/input ports of
each one of the directors boards of the plurality of second director boards;
(e) wherein the first and second directors control data transfer between the
first directors and the second directors in response to messages passing between
the first directors and the second directors through the message network to facilitate
data transfer between first directors and the second directors with such data passing
through the cache memory in the data transfer section.
Description
BACKGROUND OF THE INVENTION
This invention relates generally to data storage systems, and more particularly
to data storage systems having redundancy arrangements to protect against total
system failure in the event of a failure in a component or subassembly of the storage system.
As is known in the art, large host computers and servers (collectively referred
to herein as "host computer/servers") require large capacity data storage systems.
These large computer/servers generally includes data processors, which perform
many operations on data introduced to the host computer/server through peripherals
including the data storage system. The results of these operations are output to
peripherals, including the storage system.
One type of data storage system is a magnetic disk storage system. Here a bank
of disk drives and the host computer/server are coupled together through an interface.
The interface includes "front end" or host computer/server controllers (or directors)
and "back-end" or disk controllers (or directors). The interface operates the controllers
(or directors) in such a way that they are transparent to the host computer/server.
That is, data is stored in, and retrieved from, the bank of disk drives in such
a way that the host computer/server merely thinks it is operating with its own
local disk drive. One such system is described in U.S. Pat. No. 5,206,939, entitled
"System and Method for Disk Mapping and Data Retrieval", inventors Moshe Yanai,
Natan Vishlitzky, Bruno Alterescu and Daniel Castel, issued Apr. 27, 1993, and
assigned to the same assignee as the present invention.
As described in such U.S. patent, the interface may also include, in addition
to the host computer/server controllers (or directors) and disk controllers (or
directors), addressable cache memories. The cache memory is a semiconductor memory
and is provided to rapidly store data from the host computer/server before storage
in the disk drives, and, on the other hand, store data from the disk drives prior
to being sent to the host computer/server. The cache memory being a semiconductor
memory, as distinguished from a magnetic memory as in the case of the disk drives,
is much faster than the disk drives in reading and writing data.
The host computer/server controllers, disk controllers and cache memory are interconnected
through a backplane printed circuit board. More particularly, disk controllers
are mounted on disk controller printed circuit boards. The host computer/server
controllers are mounted on host computer/server controller printed circuit boards.
And, cache memories are mounted on cache memory printed circuit boards. The disk
directors, host computer/server directors, and cache memory printed circuit boards
plug into the backplane printed circuit board. In order to provide data integrity
in case of a failure in a director, the backplane printed circuit board has a pair
of buses. One set the disk directors is connected to one bus and another set of
the disk directors is connected to the other bus. Likewise, one set the host computer/server
directors is connected to one bus and another set of the host computer/server directors
is directors connected to the other bus. The cache memories are connected to both
buses. Each one of the buses provides data, address and control information.
The arrangement is shown schematically in FIG. 1. Thus, the use of two buses
B
1, B
2 provides a degree of redundancy to protect against a total
system failure in the event that the controllers or disk drives connected to one
bus, fail. Further, the use of two buses increases the data transfer bandwidth
of the system compared to a system having a single bus. Thus, in operation, when
the host computer/server
12 wishes to store data, the host computer
12
issues a write request to one of the front-end directors
14 (i.e., host
computer/server directors) to perform a write command. One of the front-end directors
14 replies to the request and asks the host computer
12 for the data.
After the request has passed to the requesting one of the front-end directors
14,
the director
14 determines the size of the data and reserves space in the
cache memory
18 to store the request. The front-end director
14 then
produces control signals on one of the address memory busses B
1, B
2
connected to such front-end director
14 to enable the transfer to the cache
memory
18. The host computer/server
12 then transfers the data to
the front-end director
14. The front-end director
14 then advises
the host computer/server
12 that the transfer is complete. The front-end
director
14 looks up in a Table, not shown, stored in the cache memory
18
to determine which one of the back-end directors
20 (i.e., disk directors)
is to handle this request. The Table maps the host computer/server
12 addresses
into an address in the bank
14 of disk drives. The front-end director
14
then puts a notification in a "mail box" (not shown and stored in the cache memory
18) for the back-end director
20, which is to handle the request,
the amount of the data and the disk address for the data. Other back-end directors
20 poll the cache memory
18 when they are idle to check their "mail
boxes". If the polled "mail box" indicates a transfer is to be made, the back-end
director
20 processes the request, addresses the disk drive in the bank
22, reads the data from the cache memory
18 and writes it into the
addresses of a disk drive in the bank
22.
When data is to be read from a disk drive in bank
22 to the host computer/server
12 the system operates in a reciprocal manner. More particularly, during
a read operation, a read request is instituted by the host computer/server
12
for data at specified memory locations (i.e., a requested data block). One of the
front-end directors
14 receives the read request and examines the cache
memory
18 to determine whether the requested data block is stored in the
cache memory
18. If the requested data block is in the cache memory
18,
the requested data block is read from the cache memory
18 and is sent to
the host computer/server
12. If the front-end director
14 determines
that the requested data block is not in the cache memory
18 (i.e., a so-called
"cache miss") and the director
14 writes a note in the cache memory
18
(i.e., the "mail box") that it needs to receive the requested data block. The back-end
directors
20 poll the cache memory
18 to determine whether there
is an action to be taken (i.e., a read operation of the requested block of data).
The one of the back-end directors
20 which poll the cache memory
18
mail box and detects a read operation reads the requested data block and initiates
storage of such requested data block stored in the cache memory
18. When
the storage is completely written into the cache memory
18, a read complete
indication is placed in the "mail box" in the cache memory
18. It is to
be noted that the front-end directors
14 are polling the cache memory
18
for read complete indications. When one of the polling front-end directors
14
detects a read complete indication, such front-end director
14 completes
the transfer of the requested data which is now stored in the cache memory
18
to the host computer/server
12.
The use of mailboxes and polling requires time to transfer data between the host
computer/server
12 and the bank
22 of disk drives thus reducing the
operating bandwidth of the interface.
SUMMARY OF THE INVENTION
In accordance with the present invention, a system interface is provided. The
interface includes a plurality of first director boards. Each one of the first
director boards has: (i) a plurality of first directors; and (ii) a crossbar switch
having input/output ports coupled to the first directors on such one of the first
director boards and a pair of output/input ports. A plurality of second director
boards is provided. Each one of the second directors boards has: (i) a plurality
of second directors; and (ii) a crossbar switch having input/output ports coupled
to the second directors on such one of the second director boards and a pair of
output/input ports. A data transfer section is provided having a cache memory.
The cache memory is coupled to the plurality of first and second directors. A message
network is provided, such network being operative independently of the data transfer
section. The first and second directors control data transfer between the first
directors and the second directors in response to messages passing between the
first directors and the second directors through the message network to facilitate
data transfer between first directors and the second directors. The data passes
through the cache memory in the data transfer section.
In one embodiment, each one of the directors includes: a data pipe coupled between
an input of such one of the first directors and the cache memory; a controller
for transferring the messages between the message network and such one of the first directors.
In one embodiment, each one of the directors includes a data pipe coupled between
an input of such one of the first directors and the cache memory; a microprocessor.
A controller is coupled to the microprocessor and the data pipe for controlling
the transfer of the messages between the message network and such one of the first
directors and for controlling the data between the input of such one of the first
directors and the cache memory.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other features of the invention will become more readily apparent
from the following detailed description when read together with the accompanying
drawings, in which:
FIG. 1 is a block diagram of a data storage system according to the PRIOR ART;
FIG. 2 is a block diagram of a data storage system according to the invention;
FIG. 2A shows the fields of a descriptor used in the system interface of the
data storage system of FIG. 2;
FIG. 2B shows the filed used in a MAC packet used in the system interface of
the data storage system of FIG. 2;
FIG. 3 is a sketch of an electrical cabinet storing a system interface used
in the data storage system of FIG. 2;
FIG. 4 is a diagramatical, isometric sketch showing printed circuit boards providing
the system interface of the data storage system of FIG. 2;
FIG. 5 is a block diagram of the system interface used in the data storage system
of FIG. 2;
FIG. 6 is a block diagram showing the connections between front-end and back-end
directors to one of a pair of message network boards used in the system interface
of the data storage system of FIG. 2;
FIG. 7 is a block diagram of an exemplary one of the director boards used in
the system interface of he data storage system of FIG. 2;
FIG. 8 is a block diagram of the system interface used in the data storage system
of FIG. 2;
FIG. 8A is a diagram of an exemplary global cache memory board used in the system
interface of FIG. 8;
FIG. 8B is a diagram showing a pair of director boards coupled between a pair
of host processors and global cache memory boards used in the system interface
of FIG. 8;
FIG. 8C is a block diagram of an exemplary crossbar switch used in the front-end
and rear-end directors of the system interface of FIG. 8;
FIG. 9 is a block diagram of a transmit Direct Memory Access (DMA) used in the
system interface of the FIG. 8;
FIG. 10 is a block diagram of a receive DMA used in the system interface of
FIG. 8;
FIG. 11 shows the relationship between FIGS. 11A and 11B, such FIGS. 11A and
11B together showing a process flow diagram of the send operation of a message
network used in the system interface of FIG. 8;
FIGS. 11C-11E are examples of digital words used by the message network in
the system interface of FIG. 8;
FIG. 11F shows bits in a mask used in such message network;
FIG. 11G shows the result of the mask of FIG. 11F applied to the digital word
shown in FIG. 11E;
FIG. 12 shows the relationship between FIGS. 12A and 12B, such FIGS. 12A and
12B Showing a process flow diagram of the receive operation of a message network
used in the system interface of FIG. 8;
FIG. 13 shows the relationship between FIGS. 11A and 11B, such FIGS. 11A and
11B together showing a process flow diagram of the acknowledgement operation of
a message network used in the system interface of FIG. 8;
FIGS. 14A and 14B show process flow diagrams of the transmit DMA operation
of the transmit DMA of FIG. 9; and
FIGS. 15A and 15B show process flow diagrams of the receive DMA operation of
the receive DMA of FIG. 10.
DETAILED DESCRIPTION
Referring now to FIG. 2, a data storage system
100 is shown for
transferring data between a host computer/server
120 and a bank of disk
drives
140 through a system interface
160. The system interface
160
includes: a plurality of, here 32 front-end directors
1801-
18032
coupled to the host computer/server
120 via ports-
12332;
a plurality of back-end directors
2001-
20032 coupled
to the bank of
