Method and system for nesting of communications packets Number:7,154,905 from the United States Patent and Trademark Office (PTO) owispatent A serial communications architecture for communicating between hosts and data store devices. The Storage Link architecture is specially adapted to support communications between multiple hosts and storage devices via a switching network, such as a storage area network. The Storage Link architecture specifies various communications techniques that can be combined to reduce the overall cost and increase the overall performance of communications. The Storage Link architecture may provide packet ordering based on packet type, dynamic segmentation of packets, asymmetric packet ordering, packet nesting, variable-sized packet headers, and use of out-of-band symbols to transmit control information as described below in more detail. The Storage Link architecture may also specify encoding techniques to optimize transitions and to ensure DC-balance.<H communications, packets, Method, and, syst, 7154905.html, Patent, pto, 7154905

Title: Method and system for nesting of communications packets

Abstract: A serial communications architecture for communicating between hosts and data store devices. The Storage Link architecture is specially adapted to support communications between multiple hosts and storage devices via a switching network, such as a storage area network. The Storage Link architecture specifies various communications techniques that can be combined to reduce the overall cost and increase the overall performance of communications. The Storage Link architecture may provide packet ordering based on packet type, dynamic segmentation of packets, asymmetric packet ordering, packet nesting, variable-sized packet headers, and use of out-of-band symbols to transmit control information as described below in more detail. The Storage Link architecture may also specify encoding techniques to optimize transitions and to ensure DC-balance.

Patent Number: 7,154,905 Issued on 12/26/2006 to Shin, et al.

Inventors: Shin; Yeshik (Sunnyvale, CA), Lee; David D. (Palo Alto, CA), Jeong; Deog-Kyoon (Seoul, KR), Kong; Shing (Brisbane, CA)
Assignee: Silicon Image (Sunnyvale, CA)

Appl. No.: 10/035,911

Filed: November 7, 2001

Current U.S. Class: 370/465 ; 370/468

Current International Class: H04J 15/00 (20060101)

Field of Search: 370/465,468,473,493

References Cited [Referenced By]

U.S. Patent Documents


4425666	January 1984	Groth, Jr.
4658396	April 1987	Barden
4672639	June 1987	Tanabe et al.
4797850	January 1989	Amitai
4945548	July 1990	Iannarone et al.
4972314	November 1990	Getzinger et al.
5307314	April 1994	Lee
5343473	August 1994	Cidon et al.
5400323	March 1995	Frenzel, III et al.
5430485	July 1995	Lankford et al.
5497371	March 1996	Ellis et al.
5539696	July 1996	Patel
5557608	September 1996	Calvignac et al.
5561618	October 1996	Dehesh
5574849	November 1996	Sonnier et al.
5583872	December 1996	Albrecht et al.
5587709	December 1996	Jeong
5588025	December 1996	Strolle et al.
5592487	January 1997	Knecht et al.
5663724	September 1997	Westby
5699367	December 1997	Haartsen
5703875	December 1997	Burnett
5751947	May 1998	Arakawa
5825824	October 1998	Lee et al.
5838960	November 1998	Harriman, Jr.
5872777	February 1999	Brailean et al.
5963703	October 1999	Higurashi et al.
5974464	October 1999	Shin et al.
5983287	November 1999	Kuroiwa et al.
5999571	December 1999	Shin et al.
6016510	January 2000	Quattromani et al.
6026124	February 2000	Lee et al.
6067301	May 2000	Aatresh
6151334	November 2000	Kim et al.
6154464	November 2000	Feuerstraeter et al.
6188337	February 2001	Soljanin
6188670	February 2001	Lackman et al.
6198413	March 2001	Widmer
6229859	May 2001	Jeong et al.
6266335	July 2001	Bhaskaran
6400730	June 2002	Latif et al.
6415353	July 2002	Leung
6430230	August 2002	Cunningham et al.
6438128	August 2002	Kashyap
6470391	October 2002	Takamoto et al.
6487171	November 2002	Honig et al.
6560290	May 2003	Ahn et al.
6560728	May 2003	Merritt
6587525	July 2003	Jeong et al.
6594325	July 2003	Li
6594329	July 2003	Susnow
6631132	October 2003	Sourani
6736821	May 2004	Wilson et al.
6771192	August 2004	Lee et al.
6791952	September 2004	Lin et al.
6826199	November 2004	Davidsson et al.
6891845	May 2005	Boock et al.
2002/0035633	March 2002	Bose et al.
2002/0041650	April 2002	Richmond
2002/0049825	April 2002	Jewett et al.
2002/0122420	September 2002	Yuan et al.
2002/0154633	October 2002	Shin et al.
2002/0157054	October 2002	Shin et al.
2002/0159450	October 2002	Shin et al.
2002/0159552	October 2002	Shin et al.
2002/0167945	November 2002	Shin et al.
2002/0171741	November 2002	Tonkin et al.
2002/0172200	November 2002	Shin et al.
2002/0191475	December 2002	Lee et al.
2002/0191603	December 2002	Shin et al.
2003/0140300	July 2003	Chen et al.
2003/0189922	October 2003	Howe
2003/0208655	November 2003	Lee et al.
2003/0210162	November 2003	Shin et al.
2003/0227947	December 2003	Shin et al.
2004/0004975	January 2004	Shin et al.
2005/0018697	January 2005	Enns et al.

Foreign Patent Documents


0582537	Sep., 1994	EP
0582537	Sep., 1994	EP
WO-97/13347	Apr., 1997	WO
WO-97/36408	Oct., 1997	WO
WO-99/12306	Mar., 1999	WO
WO-02/058318	Jul., 2002	WO

Other References

Richmond, Serial protocol elastic buffer description, Jul. 11, 2000, 60/217520, pp. 1-13. cited by other.

Primary Examiner: To; Doris H.
Assistant Examiner: Blount; Steve
Attorney, Agent or Firm: Perkins Coie LLP

Parent Case Text

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No. 60/252,724 entitled "METHOD AND APPARATUS FOR STORAGE I/O WITH FULL-DUPLEX ONE-TIME BLOCK I/O TRANSFER AND ADAPTIVE PAYLOAD SIZING," filed Nov. 22, 2000, and is related to U.S. patent application Ser. No. 10/037,168 entitled "METHOD AND SYSTEM FOR PLESIOSYNCHRONOUS COMMUNICATIONS WITH NULL INSERTION AND REMOVAL"; U.S. patent application Ser. No. 10/045,393, entitled "METHOD AND SYSTEM FOR TRANSITION-CONTROLLED SELECTIVE BLOCK INVERSION COMMUNICATIONS"; U.S. patent application Ser. No. 10/035,591 entitled "COMMUNICATIONS ARCHITECTURE FOR STORAGE-BASED DEVICES"; U.S. patent application Ser. No. 10/036,135 entitled "METHOD AND SYSTEM FOR PACKET ORDERING BASED ON PACKET TYPE"; U.S. patent application Ser. No. 10/036,794 entitled "METHOD AND SYSTEM FOR HOST HANDLING OF COMMUNICATIONS ERRORS"; U.S. patent application Ser. No. 10/045,606 entitled "METHOD AND SYSTEM FOR DYNAMIC SEGMENTATION OF COMMUNICATIONS PACKETS"; U.S. patent application Ser. No. 10/045,348 entitled "METHOD AND SYSTEM FOR ASYMMETRIC PACKET ORDERING BETWEEN COMMUNICATIONS DEVICES"; U.S. patent application Ser. No. 10/053,461 entitled "METHOD AND SYSTEM FOR COMMUNICATING CONTROL INFORMATION VIA OUT-OF-BAND SYMBOLS"; U.S. patent application Ser. No. 10/045,625 entitled "METHOD AND SYSTEM FOR INTEGRATING PACKET TYPE INFORMATION WITH SYNCHRONIZATION SYMBOLS"; U.S. patent application Ser. No. 10/035,911 entitled "METHOD AND SYSTEM FOR NESTING OF COMMUNICATIONS PACKETS"; U.S. patent application Ser. No. 10/045,297 entitled "COMMUNICATIONS ARCHITECTURE FOR MEMORY-BASED DEVICES"; U.S. patent application Ser. No. 10/045,600 entitled "METHOD AND SYSTEM FOR DC-BALANCING AT THE PHYSICAL LAYER"; and U.S. patent application Ser. No. 10/045,601 entitled "MULTISECTION MEMORY BANK SYSTEM", which are all hereby incorporated by reference in their entirety.

Claims

We claim:

1. A method for transmitting packets, the method comprising: receiving a first packet; transmitting a portion of the received first packet; receiving a second packet; upon receiving the second packet, stopping the transmitting of the first packet so that not all of the first packet has been transmitted; transmitting a preempt indicator; transmitting the second packet; and upon completion of transmitting the second packet, transmitting a continue indicator; and transmitting the portion of the first packet that has not yet been transmitted; wherein the packets include in-band symbols and the indicators include one or more out-of-band symbols.

2. The method of claim 1 wherein the transmitted indicators are primitives.

3. The method of claim 1 wherein the in-band symbols are transition optimized and the out-of-band symbols are not transition optimized.

4. The method of claim 1 wherein the first packet is a data packet and the second packet is a control packet.

5. The method of claim 1 wherein the first and second packets are transmitted through the same communications link.

6. The method of claim 1 wherein the transmitting of the second packet is preempted so that a third packet can be transmitted.

7. The method of claim 1 wherein the first packet includes a header that is transmitted only once.

8. A method for receiving packets, the method comprising: receiving a first portion of symbols of a first packet; receiving a preempt indicator indicating that a second packet of symbols is to be received; receiving the second packet of symbols; receiving a continue indicator indicating that a second portion of symbols of the first packet is to be received; and receiving the second portion of symbols of the first packet; wherein the symbols of the packets include in-band symbols and the indicators include one or more out-of-band symbols.

9. The method of claim 8 wherein the received indicators are primitives.

10. The method of claim 8 wherein the in-band symbols are transition optimized and the out-of-band symbols are not transition optimized.

11. The method of claim 8 wherein the first packet is a data packet and the second packet is a control packet.

12. The method of claim 8 wherein the first and second packets are received via the same communications link.

13. The method of claim 8 wherein the receiving of the second packet is preempted so that a third packet can be received.

14. The method of claim 8 wherein the first packet includes a header that is received only once.

15. A method for transmitting packets via a communications link, the method comprising: transmitting a first portion of a first packet; transmitting a preempt indicator indicating that a second packet is to be transmitted; transmitting the second packet; transmitting a continue indicator indicating that a second portion of the first packet is to be transmitted; and transmitting the second portion of the first packet wherein the first and second packets and the preempt and continue indicators are transmitted via the same communications link and wherein further the packets include in-band symbols and the indicators include one or more out-of-band symbols.

16. The method of claim 15 wherein the indicators are primitives.

17. The method of claim 15 wherein the in-band symbols are transition optimized and the out-of-band symbols are not transition optimized.

18. The method of claim 15 wherein the first packet is a data packet and the second packet is a control packet.

19. The method of claim 15 wherein the transmitting of the second packet is preempted so that a third packet can be transmitted.

20. The method of claim 15 wherein the first packet includes a header that is transmitted only once.

21. A communications device for transmitting packets via a communications link, comprising: a transmission component that transmits a first packet; and a preemption component that signals the transmission component to stop transmitting the first packet, transmits a preempt indicator indicating that a second packet is to be transmitted, transmits the second packet, and signals the transmission component to continue transmitting the first packet; wherein packets include in-band symbols and the indicators include one or more out-of-band symbols.

22. The communications device of claim 21 wherein the indicators are primitives.

23. The communications device of claim 21 wherein the in-band symbols are transition optimized and the out-of-band symbols are not transition optimized.

24. The communications device of claim 21 wherein the first packet is a data packet and the second packet is a control packet.

25. The communications device of claim 21 wherein the transmitting of the second packet is preempted so that a third packet can be transmitted.

26. The communications device of claim 21 wherein the first packet includes a header that is transmitted only once.

Description

BACKGROUND

The described technology relates generally to communications techniques and particularly to communications between hosts and data store devices.

The speed and capacity of the data store devices, such as disk drives and memories, have increased significantly over the past several years. As a result of their improved performance, these data store devices are being used in many new applications, such as database servers, Web servers, personal video recorders, and digital displays. These applications often require large amounts of data to be communicated between data store accessing devices ("hosts") and data store devices. (Hosts may include computers, CPUs, or any logic for accessing a data store device.) Moreover, as host speed increases, the speed in communicating between hosts and data store devices can have a significant impact on the overall performance of the application. In particular, even though the speed of hosts and data store devices has increased significantly, the speed of communications between hosts and data store devices has not increased as significantly, especially for communications over long distances (e.g., greater than one meter). Thus, the communications speed presents a bottleneck in many new applications.

Current communications techniques typically communicate between hosts and certain types of data store devices, such as disk drives, using a bus with many parallel lines or using a single serial communications link. The Integrated Disk Electronics ("IDE") bus and the Small Computer Systems Interface ("SCSI") bus are examples of bus-based parallel communications techniques. These communications techniques, however, present many problems. Performance of bus-based communications techniques is generally improved by increasing the number of lines in the bus, which may significantly increase the cost of such techniques. In addition, bus-based communications techniques generally provide arbitration so that multiple hosts and data store devices can share the same bus. The use of arbitration can significantly increase the cost of such a bus. The cost of such bus-based communication techniques is further increased because their design needs to address additional problems such as cross-talk and clock skew. In particular, as the communications speed increases, the solution to cross-talk and clock skew become much more complex.

Some serial communications techniques have been developed to address some of the problems of bus-based communications techniques. Current serial communications techniques, however, have problems of their own. Serial AT, Attachment, which is intended to replace IDE, does not scale well and only operates in a half duplex mode. Fibre Channel, currently used to support storage area networks ("SANs"), is very generic and therefore, not optimized for any particular application. In particular, Fibre Channel has a relatively small packet size with a large header. As a result, use of Fibre Channel often results in an unacceptably large overhead. For example, data transmitted to disk drives is typically sent in very large blocks (e.g., 216 bytes). With Fibre Channel, such large blocks need to be divided in many (e.g., 32) packets, which results in a high overhead in the amount of redundant header information and in the redundant processing performed as a packet is routed to its destination. Thus, Fibre Channel may not be appropriate for many applications.

Current memory devices, such as SDRAM and RDRAM, are typically designed to be synchronous with the accessing processing unit. The hosts and the memory devices are synchronous in that they share the same clock signal. These memory devices are typically optimized for access patterns that are both temporally and spatially related. In particular, these memory devices are optimized to read and write arrays (or streams) of data. There is a setup overhead (e.g., 5 clock cycles) when accessing the first word of an array in memory, but access of subsequent words in the array occurs at the synchronized clock rate (e.g., 1 access per clock cycle). Since the access patterns of central processing units and graphics processors are typically temporally and spatially related, they can access such memory devices efficiently.

Existing memory devices that are designed to support access patterns with a high temporal and spatial relationship may not be appropriate for uses having access patterns with a lower spatial relationship. The setup overhead for each access may be too high. There are, indeed, many uses for memory devices with access patterns that are not as spatially or temporally related as those of a central processing unit or a graphics processor. For example, a switch may have a memory device in which packets of data received via an input port are stored before they are transmitted via an output port. Traditionally, switches used crossbars to provide the switching function and FIFOs to provide a buffering function. When a memory device is used on a switch in place of a crossbar, then all the input and output ports need access to the memory device. The accesses by the different ports are, however, not particularly spatially related. Moreover, when the packet size is small (e.g., 53 bytes in the case of an ATM switch), the spatial relationship of accesses by a single port may not be significant. Other uses in which there may not be a significant spatial relationship of accesses include network processors and caches for storage area networks. In such uses, the data is received from disparate sources at disparate times and may not be spatially related.

Many existing memory devices are not particularly suitable for many uses because the memory devices typically allow access by only one accessing device at a time and because the memory devices typically operate at different clock rates than the accessing devices. Because such memory devices can only be accessed by one device at a time, the accessing devices may need to enter a wait state because the memory device is busy or a memory controller may need to have a buffering component. Of course, the use of a wait state may result in unacceptable performance. Also, the addition of a buffering component may increase complexity and cost. In addition, when multiple accessing devices access the same memory device through a single bus (e.g., one writing to the memory device and the other reading from the memory device), then all the devices that access the memory device need to be synchronized with the memory device. Because the accessing devices may have different underlying clock rates, complex and costly logic is needed to support the mapping to the bus clock rate.

Existing communications protocols, such as Fibre Channel, may have an unacceptable overhead for communicating with memory devices. The communications from a host to a memory device may occur in relatively short blocks (e.g., 32 bytes). Each block needs to be transmitted in a separate packet with a relatively large header. In some packets, the header may be larger than the data itself, which can significantly reduce the overall bandwidth and speed of transmission. More generally, communications between devices typically occurs in a synchronous or an asynchronous mode. In a synchronous mode, the transmitting and receiving devices use the same clock signal. The transmitting device can send the clock signal to the receiving device either as a separate signal or as a signal that can be derived from the data signals. When the clock is sent as a separate signal, problems arise resulting from the different delays in the data signals and the clock signal. These delays and resulting problems are increased as the transmission speed and distance are increased. It is very difficult and costly to account for these delays. In addition, the receiving device will have an asynchronous clock boundary. That is, a portion of the receiving device will operate at the clock frequency based on the transmitting device's clock frequency (i.e., the transmitter's clock domain) and another portion will operate at the receiving device's local clock frequency (i.e., the receiver's clock domain). As a result of the asynchronous boundary, the receiving device typically needs to buffer control and data signals sent between the clock domains using elastic buffers, which adds to the complexity and cost of the receiving devices. These elastic buffers require substantial space (e.g., chip area), and when a single chip has multiple communications ports, the design is complicated because each port needs its own elastic buffer. When the clock is derived from the data signal, the problems of the delay are reduced somewhat, but there are still the problems associated with an asynchronous clock boundary.

A plesiosynchronous clocking technique can be used to avoid the need to transmit a separate clock signal or derive the clock signal from the data signal. With plesiosynchronous clocking (also known as "plesiochronous" clocking), the transmitting and receiving devices have clocks with nominally the same clock frequency. If the clock frequencies were exactly the same, then transmitting and receiving devices would be synchronized and the receiving device could accurately identify the transmitted data (in the case of serial transmission). Also, since the receiving device operates only at its local clock frequency, there is no asynchronous clock boundary. In practice, however, clock frequencies are not exactly the same but vary, for example, by 100 ppm. The receiving device can use techniques as described in U.S. Pat. No. 6,229,859, entitled "System and Method for High-Speed, Synchronized Data Communication," which is hereby incorporated by reference, to account for clock variations. Those techniques use an oversampling of the data by the receiving device to detect edge boundaries of the transmitted data. The receiving device can vary the number of bits of data detected during an interval to compensate for the variations in frequency.

It would be desirable to have a communications architecture that provides high-performance for applications (e.g., data storage-based applications and memory-based applications) at a low cost. Such a communications architecture would allow for communications techniques to be tailored to particular applications.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating components of the Storage Link architecture in one embodiment.

FIG. 2 is a block diagram illustrating components of the transport layer, link layer, and physical layer in one embodiment.

FIG. 3 is a block diagram illustrating components of the physical layer in one embodiment.

FIG. 4 is a block diagram illustrating the format of a packet in one embodiment.

FIG. 5 is a block diagram illustrating the organization of a data packet and a segmented data packet in one embodiment.

FIG. 6 is a block diagram illustrating a receive packet memory before and after merging of a received packet.

FIG. 7 is a logic diagram illustrating the segmentation of a data packet by the transport layer in one embodiment.

FIG. 8 is a logic diagram illustrating the merging of data packets by the transport layer in one embodiment.

FIG. 9A is a diagram illustrating the use of a synchronization primitive encoded with the packet type in one embodiment.

FIG. 9B illustrates the identification of synchronization symbols in one embodiment.

FIG. 9C is a block diagram illustrating a component of a link layer for inserting a synchronization symbol into a stream of symbols.

FIG. 10 is a block diagram illustrating an organization of a packet memory in one embodiment.

FIG. 11 is a logic diagram illustrating the process of storing packets in a transmit packet memory in one embodiment.

FIG. 12 is a logic diagram illustrating the transport layer processing of packets stored in the packet memory in one embodiment.

FIG. 13 is a diagram illustrating the preemption of a packet in one embodiment.

FIG. 14 is a logic diagram illustrating the processing of packet preemption by a transmitting communications node in one embodiment.

FIG. 15 is a logic diagram illustrating the processing of packet preemption by a receiving communications node in one embodiment.

FIG. 16 is a block diagram illustrating asymmetric packet ordering in one embodiment.

FIG. 17 is a diagram illustrating transaction-based asymmetric packet ordering in one embodiment.

FIG. 18 is a logic diagram illustrating processing by a switch that ensures packet ordering within a transaction in one embodiment.

FIG. 19A is a block diagram illustrating error handling during transmission from a host to a data store device in one embodiment.

FIG. 19B is a block diagram illustrating the handling of errors during the transmission from a data store device to a host in one embodiment.

FIG. 19C is a logic diagram illustrating the processing of a communications node detecting an error in one embodiment.

FIG. 20 illustrates an alternate bit inversion encoding technique in one embodiment.

FIG. 21A is a diagram illustrating the selective block inversion encoding technique in one embodiment.

FIG. 21B is a block diagram illustrating selective block inversion with a programmable block size in one embodiment.

FIG. 21C is a block diagram illustrating transition inversion and polarity inversion in one embodiment.

FIG. 22 is a diagram illustrating the format of a primitive in one embodiment.

FIG. 23 is a logic diagram illustrating transition control and selective block inversion encoding in one embodiment.

FIG. 24 is a logic diagram illustrating transition control optimization in one embodiment.

FIG. 25 is a logic diagram illustrating block inversion in one embodiment.

FIG. 26 is a logic diagram illustrating transition control and selective block inversion decoding in one embodiment.

FIG. 27 is a logic diagram illustrating the undoing of block inversion in one embodiment.

FIG. 28 is a logic diagram illustrating the undoing of transition control optimization in one embodiment.

FIG. 29 is a logic diagram illustrating the processing of a primitive in one embodiment.

FIG. 30 is a block diagram illustrating the overall architecture of a multiport memory device in one embodiment.

FIG. 31 is a block diagram illustrating the use of a single phase lock loop for the physical layer of a multiport memory device in one embodiment.

FIG. 32 is a block diagram illustrating contents of a bank cache in one embodiment.

FIG. 33 is a logic diagram illustrating processing of the access layer when it receives data from a host in one embodiment.

FIG. 34 is a logic diagram illustrating the processing of a write command by the access layer in one embodiment.

FIG. 35 is a logic diagram illustrating processing of the access layer when it is to provide data to the transport layer in one embodiment.

FIG. 36 is a block diagram of a multisection memory bank in one embodiment.

FIG. 37A is a block diagram illustrating a variable strength line driver in one embodiment.

FIG. 37B is a block diagram illustrating the circuit of a variable-strength line driver in one embodiment.

FIG. 38A is a block diagram of a plesiosynchronous communications system in one embodiment.

FIG. 38B is a block diagram illustrating a switch supporting multiple data store devices in one embodiment.

FIG. 39A is a block diagram illustrating a prior art synchronous clocking system.

FIG. 39B is a block diagram illustrating resolution of an asynchronous clock boundary in the physical layer in one embodiment.

FIG. 40 is a block diagram illustrating NULL insertion and deletion at the physical layer in one embodiment.

FIG. 41 is a block diagram illustrating the frame aligner in one embodiment.

FIGS. 42A, 42B, and 42C illustrate the loading of the bit buffer in one embodiment.

FIG. 43 is a diagram illustrating the detection of a synchronization primitive by the sync and null detector in one embodiment.

FIG. 44 is a diagram illustrating the portion of the bit buffer that contains the next symbol to be extracted in one embodiment.

FIG. 45 is a state diagram of the symbol pointer tracker in one embodiment.

FIG. 46 is a diagram illustrating movement of the start-of-symbol pointer due to overrun or underrun in one embodiment.

FIGS. 47A and 47B are diagrams illustrating NULL detection while the start-of-symbol pointer is not within the overrun or underrun extend regions in one embodiment.

FIGS. 48A and 48B are diagrams illustrating NULL detection while the start-of-symbol pointer is in the underrun extend region.

FIGS. 49A and 49B are diagrams illustrating NULL detection while the start-of-symbol pointer is in the overrun extend region.

DETAILED DESCRIPTION

A serial communications architecture for communicating between hosts and data store devices is provided. Aspects of the serial communications architecture, referred to as "Storage Link," are specially adapted for communications with storage devices, such as disk drives. Other aspects of the serial communications architecture, referred to as "Memory Link," are specially adapted for communications with memory devices, such as RAM devices. In addition, Storage Link and Memory Link may use an improved plesiosynchronous technique when receiving data transmitted via a serial communications link.

In one embodiment, the Storage Link architecture is specially adapted to support communications between multiple hosts and storage devices via a switching network, such as a storage area network. The Storage Link architecture specifies various communications techniques that can be combined to reduce the overall cost and increase the overall performance of communications. The Storage Link architecture defines a hierarchy of transport, link, and physical layers such that each layer in the hierarchy is aware of and can take advantage of characteristics of lower layers in the hierarchy. For example, the transport layer, which is responsible for handling packets, may receive signals from the link layer that indicate when an end of packet is received. Traditional communications architectures prohibit a lower layer from being aware of the functions of a higher layer. The Storage Link architecture may provide packet ordering based on packet type, dynamic segmentation of packets, asymmetric packet ordering, packet nesting, variable-sized packet headers, and use of out-of-band symbols to transmit control information as described below in more detail. The Storage Link architecture may also specify encoding techniques to optimize transitions and to ensure DC-balance. The Storage Link architecture may also use the NULL insertion and removal techniques with plesiosynchronous clocking as described below in detail. The Storage Link architecture thus provides an improved way to access storage devices that reduces cost and increases communications speed.

In one embodiment, the Memory Link architecture also provides various communications techniques that can be combined to reduce the overall cost and increase the overall performance of communications between a host and a memory device. Like the Storage Link architecture, the Memory Link architecture uses a hierarchy of layers in which each layer can take advantage of the characteristics of a lower layer in the hierarchy. The Memory Link architecture in one embodiment provides a multiport memory device for serial communications. Each port may use a plesiosynchronous technique when receiving data and may share the same phase lock loop to control sampling of the received data. The Memory Link architecture may also use a physical layer D-C balancing technique that provides an additional driving of a communications link to offset the running disparity. The Memory Link architecture may also use an improved plesiosynchronous clock technique that inserts and removes special symbols from the received symbols to compensate for variations in clock frequencies between a transmitting and receiving device. Various combinations of the Memory Link architecture allow for the cost of designing, developing, and manufacturing memory-based systems to be reduced and the speed of memory access to be increased.

In one embodiment, the plesiosynchronous clocking technique uses the insertion and removal of symbols by the physical layer of a receiving communications node to compensate for variations in clock frequency between the transmitter and receiver. The receiver maintains a small buffer (e.g., 27 bits of the received data) at the physical layer. When the receiver detects a synchronization symbol, it initializes a start-of-symbol pointer in the buffer. As subsequent bits of data are received, the start-of-symbol pointer is adjusted to account for the variation in clock frequencies of the transmitter and the receiver. In particular, if the transmitter's clock frequency is faster ("an overrun condition"), then the pointer is progressively moved backward in the buffer (i.e., toward the last received bits). Analogously, if the transmitter's clock frequency is slower ("an underrun condition"), then the pointer is progressively moved forwards in the buffer (i.e., toward the first received bits). When the receiver detects a special symbol of a certain type (e.g., a NULL symbol) and the pointer has moved to near the beginning or end of the buffer, then the receiver removes the special symbol from the buffer to account for the overrun condition or inserts a special symbol into the buffer to account for the underrun condition. In this way, the receiver stays synchronized with the transmitter even though their clock frequencies vary. Also, asynchronous clock boundaries are avoided in the receivers.

In the following, aspects of the serial communications architecture are described using block diagrams and logic diagrams. One skilled in the art will appreciate that the serial communications architecture can be implemented using different combinations of logic circuits and/or firmware. In particular, the logic diagrams illustrate processing that may be performed in parallel using duplicate logic circuits (e.g., one for each communications link) or may be performed in serial using a single logic circuit. The particular logic designs can be tailored to meet the cost and performance objectives of the implementation of the serial communications architecture. One skilled in the art will be able to readily design logic circuits based on the following descriptions.

1. Storage Link Architecture

FIG. 1 is a block diagram illustrating components of the Storage Link architecture in one embodiment. The hosts 110 and data store devices 120 are interconnected to switching network 130 via serial communications links 140. The hosts may include computer systems that access the data store devices. The data store devices may include storage area network devices (e.g., disk drives), high-speed memory devices, and other devices for storing data. The Storage Link architecture, however, is designed to support block-oriented data store devices, such as disk drives. The switching network may include multiple switches that are interconnected so that communications paths, especially between hosts and data store devices, can be established.

Each host, data store device, and switch is a communications node that includes one or more communications interfaces 115, 125, and 135 with a transport layer, a link layer, and a physical layer. The hosts and data store devices have an upper layer that communicate to each other via transactions. When transmitting data, the upper layer provides the data for the transaction to the transport layer. The transport layer receives the data of the transaction and generates packets (i.e., packetizes the data) for transmission. The transport layer then provides each code (e.g., each byte) of the packets to the link layer for transmission to the destination. When receiving data, the transport layer receives the codes of transaction from the link layer and identifies the packets. The transport layer then combines the data of the packets of the transaction (i.e., depacketizes the data) and provides the data of the transaction to the upper layer. Each host and data store device may include an application upper layer that provides the data of a transaction to the transport layer to be transmitted to a destination and receives the data of a transaction from the transport layer that was transmitted by a source. Each packet includes a header section and a payload section. The header section identifies the source and destination and a packet type (e.g., data or command). Each switch may include a communications interface for each port of the switch. Each port may be connected to a serial communications link comprising a receive and transmit link. The switches and their interconnecting communications links form the switching network. The transport layer of a switch may not packetize and depacketize the data, rather it may direct packets received via one port to be transmitted via another port to affect the routing of the packets from the source to the destination as indicated in the header section.

The link layer encodes the codes to be transmitted via its communications link into symbols and decodes symbols received via its communications link into codes. The link layer receives codes to be transmitted from the transport layer, encodes those codes into symbols, and provides those symbols to the physical layer. The link layer also receives symbols from the physical layer, decodes those symbols into codes, and provides those codes to the transport layer. The encoding may include mapping the codes to symbols to optimize bit transitions and to ensure DC-balance. The physical layer receives the symbols from the link layer, serializes the symbols, and transmits the serialized symbols via the communications link. The physical layer receives serialized symbols via the communications link, deserializes the symbols, and provides the symbols to the link layer.

FIG. 2 is a block diagram illustrating components of the transport layer, link layer, and physical layer in one embodiment. Each layer includes transmit components 201 and receive components 202. The transport layer includes transmit components, such as a transmit memory 211, a packetizer 212, and a transmit controller 213, and receive components, such as a receive memory 215, a depacketizer 216, and a receive controller 217. The data of the transaction to be transmitted is stored in the transmit memory. The transmit controller controls the packetizer to retrieve data of a transaction from the transmit memory, packetize the data, and provide each code of the packets to the link layer. The receive controller controls the depacketizer to receive codes provided by the link layer, identify the packets, and store the codes as packets in the receive memory.

The link layer includes transmit components, such as an encoder 221, a data selector 222, and a control symbol generator 223, and receive components, such as a decoder 225 and a control symbol identifier 226. The link layer may be responsible for generating and transmitting and for receiving and identifying, control symbols, synchronization symbols, timing symbols, and so on, as described below in more detail. In one embodiment, however, the physical layer is responsible for detecting control symbols and synchronization symbols. The transport layer may indicate to the link layer when to transmit control symbols, and the link layer may indicate to the transport layer when control symbols are received. For example, the transmit layer may notify the link layer when the receive memory is full. In such a case, the link layer may transmit a control symbol (e.g., XOFF) notifying the other end of the communications link not to transmit any more data. When such transmitted control symbol is received, the link layer of the other end may indicate to its transport layer to stop transmitting data. The data selector selects symbols encoded from codes provided by the transport layer or symbols generated by the control symbol generator. The data selector effectively inserts control symbols and synchronization symbols into the sequence of symbols that are to be transmitted. Various possible encoding techniques for the codes are described below in detail. The data decoder receives symbols from the physical layer and decodes them into codes that are provided to the transport layer. The control symbol identifier identifies control symbols and signals the transport layer as appropriate.

The physical layer includes a transmitter 231 and receiver 235. The transmitter serializes the symbols provided by the link layer and transmits the serialized symbols onto the communications link. The receiver receives the serialized symbols via the communications link, deserializes the symbols, and provides the symbols to the link layer.

The transmit components and the receive components can transmit and receive packets in full duplex mode. That is, a packet received by the transport layer can be transmitted by the transmit components simultaneously with a packet being received by the receive components. As discussed below in detail, the encoding techniques enable transmitting of control symbols by the link layer for link control (e.g., flow control) even while packets are being transmitted in full duplex mode. That is, link control symbols can be inserted into a sequence of symbols generated from codes provided by the transport layer. Thus, a communications link can be simultaneously transmitting and receiving data symbols, and control symbols can be inserted for link control. Prior transmission techniques typically transmitted in half-duplex