Switching system and methodology having scheduled connection on input and output ports responsive to common time reference Number:7,426,206 from the United States Patent and Trademark Office (PTO) owispatent An interface method and system between asynchronous data packet flows and synchronized switching systems, which utilize a global common time reference. The synchronized switching systems utilize a time frame switching method based on predefined switching schedules that are responsive to a global common time reference, where the global common time reference is divided into a plurality of contiguous periodic time frames. The asynchronous data packet flows are routed according to information contained in the packets' header. The interface method and system maps the header information of the asynchronous data packet flows to respective time frames that match the predefined switching schedule over the synchronized switching system. The interface system can aggregate multiple asynchronous data packet flows into a single pre-defined switching schedule over the synchronized switching system.<H methodology, responsive, Switching, syste, 7426206.html, Patent, pto, 7426206

Title: Switching system and methodology having scheduled connection on input and output ports responsive to common time reference

Abstract: An interface method and system between asynchronous data packet flows and synchronized switching systems, which utilize a global common time reference. The synchronized switching systems utilize a time frame switching method based on predefined switching schedules that are responsive to a global common time reference, where the global common time reference is divided into a plurality of contiguous periodic time frames. The asynchronous data packet flows are routed according to information contained in the packets' header. The interface method and system maps the header information of the asynchronous data packet flows to respective time frames that match the predefined switching schedule over the synchronized switching system. The interface system can aggregate multiple asynchronous data packet flows into a single pre-defined switching schedule over the synchronized switching system.

Patent Number: 7,426,206 Issued on 09/16/2008 to Ofek, et al.

Inventors: Ofek; Yoram (Riverdale, NY), Baldi; Mario (Cuneo, IT)
Assignee: Synchrodyne Networks, Inc. (New York, NY)

Appl. No.: 09/535,831

Filed: March 28, 2000

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
09120636	Aug., 2001	6272131
60088983	Jun., 1998

Current U.S. Class: 370/389

Current International Class: H04L 12/28 (20060101)

Field of Search: 370/229,231,352,353,354,355,357,369,375,395.43,395.62,419,503,508,389,391,392,241,252,253,351,356,464,465,504,509,516

References Cited [Referenced By]

U.S. Patent Documents


5418779	May 1995	Yemini et al.
6038230	March 2000	Ofek
6259695	July 2001	Ofek
6272131	August 2001	Ofek
6272132	August 2001	Ofek et al.
6330236	December 2001	Ofek et al.
6351474	February 2002	Robinett et al.
6377579	April 2002	Ofek
6385198	May 2002	Ofek et al.
6442135	August 2002	Ofek

Other References

Baldi End-to-end delay of videoconferencing over packet switched networks INFOCOM '98. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE , vol. 3, Mar. 29-Apr. 2, 1998. cited by examiner .
Li Distributed source-destination synchronization. Communications, 1996. ICC 96, Conference Record, Converging Technologies for Tomorrow's Applications. 1996 IEEE International Conference on , vol. 3, Jun. 23-27. cited by examiner .
A. Pattavina, "Switching Theory: Architecture and Performance in Broadband ATM Networks", John Wiley & Sons, NY (1998), table of contents. cited by other .
Nichols, et al. "Definition of the Differnetiated Services Field (DS Field) in the IPv4 and Ipv6 Headers", Network Working Group Request for Comments 2474, Dec. 1998, pp. 1-20. cited by other .
Kamiyama, et al., "Quasi-STM Transmission Method Based on ATM Network," IEEE Globecom'94, 1994, pp. 1808-1814. cited by other .
Mills, et al., "Final Report on the Highball Project," Technical Report 95-4-1, University of Delaware, Apr. 1995. cited by other .
Awdeh, et al., "Survey of ATM Switch Architectures," Computer Networks and ISDN Systems, No. 27, 1995, pp. 1567-1613. cited by other .
Broomell, et al., "Classification Categories and Historical Development of Switching Fabric Topologies," Computing Surveys, vol. 15, No. 2, Jun. 1983. cited by other .
Ahmadi, et al., "A Survey of Modern High-Performance Switching Techniques," IEEE Journal on Selected Areas in Communications, vol. 7, No. 7, Sep. 1989. cited by other .
T. G. Robertazzi, Editor, "Performance Evaluation of High Speed Switching Fabrics and Networks," IEEE Press, 1992 Table of Contents. cited by other .
Goke, et al., "Banyan Networks for Partitioning Multiprocessor Systems,"1st Annual Symposium on Computer Architecture, Dec. 1973, pp. 21-28. cited by other .
Shiomoto, et al., "Dynamic Burst Transfer Time-Slot-Base Network," IEEE Communications Magazine, Oct. 1999, pp. 88-96. cited by other .
Bohm, et al., "The DTM Gigabit Network," Journal of High Speed Networks, vol. 3, No. 2, 1994. cited by other .
Bohm, et al., "Fast Circuit Switching for the Next Generation of High Performance Networks," IEEE Journal on Selected Areas in Communications, vol. 14, No. 2, pp. 298-305, Feb. 1996. cited by other .
Y. Ofek, "Integration Of Voice Communication On A Synchronous Optical Hypergraph", IEEE INFOCOM'88, 1988. cited by other .
Li et al., "Time-Driven Priority: Flow Control For Real-Time Heterogeneous Internetworking", IEEE INFOCOM'96, 1996. cited by other .
Li et al., "Pseudo-Isochronous Cell Forwarding", IEEE INFOCOM'94, pp. 1-19; 1994. cited by other .
A. R. Jacob, "A Survey of Fast Packet Switches", Computer Communications Review, Jan. 1990, pp. 54-64. cited by other .
Y. Ofek, "The Topology, Algorithms And Analysis Of A Synchronous Optical Hypergraph Architecture", Ph.D. Dissertation, Electrical Engineering Department, University of Illinois at Urbana, Report No. UIUCDCS-R-87 1343, May 1987 pp. 84-154. cited by other .
A. G. Fraser, "Early Experiment with Asynchronous Time Division Networks", IEEE Networks, pp. 12-26, Jan. 1993. cited by other .
A. Pattavina, "Non-blocking Architecture for ATM Switching", IEEE Communications Magazine, Feb. 1993, pp. 37-48. cited by other .
John C. Bellamy, "Digital Network Synchronization", IEEE Communications Magazine, Apr. 1995, pp. 70-83. cited by other .
E. W. Zegura, "Architecture for ATM Switching Systems", IEEE Communications Magazine, Feb. 1993, pp. 28-37. cited by other .
A. Tannebaum, Computer Networks (3rd Ed.) Prentice Hall, 1996 Chapter 5. cited by other .
S.J.Golestani, "A Framing Strategy for Congestion Management", IEEE Journal on Selected Areas in Communications, vol. 9, No. 7, Sep. 1991, pp. 1064-1077. cited by other .
Li et al., "Pseudo-isonchronous Cell Forwarding", Computer Networks and ISDN Systems, vol. 30 (1998), pp. 2359-2372. cited by other .
Li et al., "Time-driven Priority' Flow Control for Real-time Heterogeneous Internetworking", Proceedings of Infocom, vol. 15, Los Alamitos, CA, Mar. 1996, pp. 189-197. cited by other.

Primary Examiner: Levitan; Dmitry
Attorney, Agent or Firm: Sitrick & Sitrick

Parent Case Text

RELATED APPLICATIONS

This application is a continuation-in-part of application Ser. No. 09/120,636, filed Jul. 22, 1998--now issued U.S. Pat. No. 6,272,131, issued Aug. 7, 2001--which claims priority from provisional patent application Ser. No. 60/088,983, filed Jun. 11, 1998.

Claims

What is claimed is:

1. An input interface system for mapping an asynchronous stream of data packets, each comprising a header portion and a payload portion, from at least one source to at least one destination, said system comprising: a Common Time Reference (CTR), divided into a plurality of contiguous periodic super cycles each comprised of at least one contiguous time cycles each comprised of at least one contiguous time frame (TF); at least one synchronous virtual pipe (SVP) having a subset of predefined time frames uniquely associated therewith; at least one queue wherein each queue is associated with a respective one of the SVPs; means for analyzing the header portions of the asynchronous data packets; means for storing the analyzed data packets in respective queues responsive to the means for analyzing; a link coupled to the destination; and an SVP Forwarding Controller, comprising a second memory for storing SVP schedules, and for forwarding, to the link, respective ones of the asynchronous data packets from respective ones of the queues responsive to the respective SVP schedule and the CTR.

2. The system as in claim 1, wherein there are a plurality of SVPs, and wherein there are a plurality of respective associated queues.

3. The system as in claim 1, wherein the data packets are forwarded out of the respective one of the queues during predefined time frames in a cyclically recurring order.

4. The system as in claim 3, wherein the cyclically recurring order is a predefined number of at least one time cycle.

5. The system as in claim 3, wherein the cyclically recurring order is a predefined number of at least one super cycle.

6. The system as in claim 3, wherein the cyclically recurring order is a summation of a predefined number of time frames plus a predefined number of time cycles plus a predefined number of super cycles.

7. The system as in claim 6, wherein the recurring order starts at an arbitrary point of time in the CTR.

8. The system as in claim 1, wherein the link is comprised of at least one of a plurality of channels; wherein the SVP Forwarding Controller provides mapping for forwarding of the respective data packets from a respective one of the queues to a respective one of the channels during selected respective ones of the time frames, responsive to the SVP schedules and the CTR.

9. The system as in claim 8, wherein the SVP Forwarding Controller is comprised of a plurality of SVP forwarding controllers.

10. The system as in claim 9, wherein each of the plurality of SVP Forwarding Controllers is associated with at least one of the channels.

11. The system as in claim 9, wherein there are a plurality of sets of queues, each set comprising at least one queue, wherein each set is associated with one respective one of the SVP Forwarding Controllers.

12. The system as in claim 1, wherein there are a plurality of separate and independent streams of asynchronous data packets.

13. The system as in claim 12, wherein there are a plurality of SVP Forwarding Controllers each associated with at least one of the plurality of asynchronous data streams.

14. The system as in claim 13, wherein there are a plurality of sets of queues, each set comprising at least one queue, wherein each set is associated with one respective one of the SVP Forwarding Controllers.

15. The system as in claim 12, wherein there are a plurality of the means for analyzing; wherein each of the means for analyzing provides analysis of at least one of the plurality of streams of asynchronous data packets.

16. The system as in claim 12, wherein there are a plurality of means for analyzing; wherein each of the plurality of streams is associated with at least one of the means for analyzing.

17. An input interface system for mapping an asynchronous stream of data packets, each comprising a header portion and a payload portion, from at least one source to at least one destination, said system comprising: a Common Time Reference (CTR), divided into a plurality of contiguous periodic super cycles each comprised of at least one contiguous time cycles each comprised of at least one contiguous time frame (TF); at least one synchronous virtual pipe (SVP) having a subset of predefined ones of the time frames uniquely associated therewith; a plurality of queues wherein each queue is associated with a respective one of the SVPs, and wherein each of the time frames is associated with one of the queues; means for analyzing the header portions of the asynchronous data packets; means for storing the analyzed data packets in respective queues responsive to the means for analyzing; a link coupled to the destination; and a forwarding controller responsive to the CTR for forwarding a respective one of the data packets from the respective one of queues that is associated with the respective time frame to the link.

18. The system as in claim 17, wherein each of the queues is subdivided into a Constant Bit Rate (CBR), a Variable Bit Rate (VBR), and a best efforts (BE) queue; wherein the means for analyzing is further comprised of a controller and a scheduling table, and provides for identifying respective ones of the data packets as CBR, VBR, and BE; wherein the means for storing provides for storage of the respective data packets in the respective CBR, VBR, and BE queues for an associated respective one of the queues associated with an associated respective one of the time frames, responsive to the means for analyzing.

19. The system as in claim 18, wherein the output from the respective ones of the queues is prioritized to provide first for output from the respective one of the queue's CBR queue, then from the respective one of the queue's VBR queue, and then from the respective one of the queue's BE queue.

20. The system as in claim 19, wherein in one case, certain ones of the data packets from the CBR, VBR, and BE queues for the respective one of the time frames are not output during a respective associated one of the time frames, the system further comprising: a rescheduling controller for detecting the one case and for rescheduling the certain ones of the data packets.

21. The system as in claim 20, wherein the rescheduling is provided responsive to the controller in the means for analyzing.

22. The system as in claim 18, wherein in one case during at least one given time frame, certain ones of the data packets from the respective SVP are not output, the system further comprising: a rescheduling controller for detecting said one case, and providing for rescheduling of the certain ones of the data packets for a next available one of the subset of time frames associated with the respective SVP.

23. The system as in claim 17, wherein the data packets are forwarded out of the respective queues during predefined one of the time frames in a cyclically recurring order.

24. The system as in claim 23, wherein the cyclically recurring order is a predefined number of at least one time cycle.

25. The system as in claim 23, wherein the cyclically recurring order is a predefined number of at least one super cycle.

26. The system as in claim 23, wherein the cyclically recurring order is a summation of a predefined number of time frames plus a predefined number of time cycles plus a predefined number of super cycles.

27. The system as in claim 17, wherein the Forwarding Controller is comprised of a plurality of forwarding controllers, wherein each of the plurality of forwarding controllers is associated with at least one of the channels.

28. The system as in claim 27, wherein there are a plurality of sets of queues, each set comprising at least one queue, wherein each set is associated with a respective one of the forwarding controllers.

29. The system as in claim 27, wherein there are a plurality of separate and independent streams of asynchronous data packets.

30. The system as in claim 29, wherein there are a plurality of means for analyzing each associated with at least one of the plurality of asynchronous data streams.

31. The system as in claim 20, wherein there are a plurality of sets of queues, each set comprising at least one queue, wherein each set is associated with a respective one of the forwarding controllers.

32. The system as in claim 31, wherein each of the means for analyzing provides analysis of at least one of the plurality of streams of asynchronous data packets.

33. The control system as in claim 17, wherein the CTR is Coordinated Universal Time (UTC) standard; and wherein the super cycle is one of a single UTC second, a predefined integer number of UTC seconds, and a fraction of one UTC second.

34. A communications system, comprising: means for mapping an asynchronous stream of data packets, each comprising a header portion and a payload portion, through an input interface system via a communications link from at least one source to at least one switching subsystem; a Common Time Reference (CTR), divided into a plurality of contiguous periodic super cycles each comprised of at least one contiguous time cycles each comprised of at least one contiguous time frame (TF); wherein the input interface subsystem is comprised of: at least one synchronous virtual pipe (SVP) having a subset of predefined time frames uniquely associated therewith; at least one queue wherein each queue is associated with a respective one of the SVPs; means for analyzing the header potions of the asynchronous data packets; means for storing the analyzed data packets in respective associated queues responsive to the means for analyzing; and an SVP Forwarding Controller, comprising a second memory for storing SVP schedules, and for forwarding, to the link, respective ones of the asynchronous data packets from respective ones of the queues responsive to the SVP schedule and the CTR.

35. The system as in claim 34, wherein there are a plurality of SVPs, and wherein there are a plurality of respective associated queues.

36. The system as in claim 34, wherein the data packets are forwarded out of the respective queues to the link during predefined time frames in a cyclically recurring order.

37. The system as in claim 36, wherein the cyclically recurring order is a predefined number of at least one time cycle.

38. The system as in claim 36, wherein the cyclically recurring order is a predefined number of at least one super cycle.

39. The system as in claim 36, wherein the cyclically recurring order is a summation of a predefined number of time frames plus a predefined number of time cycles plus a predefined number of super cycles.

40. The system as in claim 39, wherein the recurring order starts at an arbitrary point of time in the CTR.

41. The system as in claim 34, wherein the link is comprised of at least one of a plurality of channels; wherein the SVP Forwarding Controller provides mapping for forwarding of the respective data packets from a respective one of the queues to a respective one of the channels during selected respective ones of the time frames, responsive to the SVP schedules and the CTR.

42. The system as in claim 34, wherein there are a plurality of separate and independent streams of asynchronous data packets.

43. The system as in claim 42, wherein there are a plurality of SVP Forwarding Controllers each associated with at least one of the plurality of asynchronous data streams.

44. The system as in claim 34, wherein the switching subsystem is comprised of at least one input port, at least one output port, and a switching fabric for coupling a respective selected one of the input ports to a respective selected one of the output ports.

45. The system as in claim 44, wherein the at least one input port is further comprised of an alignment subsystem for aligning timing for the data packets received at the input port, relative to the CTR.

46. The system as in claim 45, wherein the alignment subsystem provides for aligning the timing for each time frame for a set comprising the data packets received at the input port during the respective one of the time frames, the system further comprising: means for scheduling the transfer of the respective set of the data packets during a subsequent time frame associated with the respective input port.

47. An input interface method comprising: mapping an asynchronous stream of data packets, each comprising a header portion and a payload portion, from at least one source to at least one destination; providing a Common Time Reference (CTR), divided into a plurality of contiguous periodic super cycles each comprised of at least one contiguous time cycles each comprised of at least one contiguous time frame (TF); providing at least one synchronous virtual pipe (SVP) having a subset of predefined ones of the time frames uniquely associated therewith; providing a plurality of queues, wherein each queue is associated with a respective one of the SVPs, and wherein each of the time frames is associated with one of the queues; analyzing the header portions of the asynchronous data packets; storing the analyzed data packets in respective queues responsive to the means for analyzing; providing a link coupled to the destination; dividing each of the queues into a Constant Bit Rate (CBR) queue, a Variable Bit Rate (VBR) queue, and a best effort (BE) queue; identifying respective ones of the data packets as CBR data packet, VBR data packet, and BE data packet; storing the respective data packets in the respective CBR, VBR, and BE queues for a respective queue associated with a respective time frame, responsive to the identifying and analyzing; and forwarding a respective one of the data packets from the respective one of queues that is associated with the respective time frame responsive to the CTR.

48. The method as in claim 47, further comprising: prioritizing the output from the respective one of the queues to provide first for output from the respective one of the queue's CBR queue, then from the respective one of the queue's VBR queue, and then from the respective one of the queue's BE queue.

49. The method as in claim 48, further comprising: determining which certain ones of the data packets from the CBR, VBR, and BE queues for the respective one of the time frames are not output during a respective associated one of the time frames; and detecting and rescheduling the certain ones of the data packets.

50. The method as in claim 47, wherein each SVP is associated with at least one Pipe ID (PID).

51. The method as in claim 50, wherein the PID is one of the following: explicitly contained in a field of the data packet header portion, implicitly given by an Internet protocol (IP) address, Internet protocol group multicast address, a combination of values in the IP address and transport control protocol (TCP), a user datagram protocol (UDP) header, an MPLS label, an asynchronous transfer mode (ATM) virtual circuit identifier (VCI), and an ATM virtual path identifier (VPI), a combination of VCI and VPI.

52. The method as in claim 47, wherein the forwarding of the data packets during each of the time frames is performed from the CBR, VBR, and BE queues.

53. The method as in claim 52, further comprising: inserting a delimiter between data packets that are transmitted within the same time frames.

54. The method as in claim 53, wherein the inserting of the delimiter is done between at least one of: CBR data packet and VBR data packet, CBR data packet and BE data packet, VBR data packet and CBR data packet, VBR data packet and BE data packet, BE data packet and CBR data packet, BE data packet and VBR data packet.

Description

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable.

BACKGROUND OF THE INVENTION

This invention relates generally to a method and apparatus for switching of data packets in a communications network in a timely manner while providing low switching complexity and performance guarantees.

Circuit-switching networks, which are still the main carrier for real-time traffic, are designed for telephony service and cannot be easily enhanced to support multiple services or carry multimedia traffic. Its almost synchronous byte switching enables circuit-switching networks to transport data streams at constant rates with little delay or jitter. However, since circuit-switching networks allocate resources exclusively for individual connections, they suffer from low utilization under bursty traffic. Moreover, it is difficult to dynamically allocate circuits of widely different capacities, which makes it a challenge to support multimedia traffic. Finally, the almost synchronous byte switching of SONET, which embodies the Synchronous Digital Hierarchy (SDH), requires increasingly more precise clock synchronization as the lines speed increases [John C. Bellamy, "Digital Network Synchronization", IEEE Communications Magazine, April 1995; pages 70-83].

Packet switching networks like IP (Internet Protocol)-based Internet and Intranets [see, for example, A. Tannebaum, Computer Networks (3rd Ed.) Prentice Hall, 1996] handle bursty data more efficiently than circuit switching, due to their statistical multiplexing of the packet streams. However, current packet switches and routers operate asynchronously and provide "best effort" service only, in which end-to-end delay and jitter are neither guaranteed nor bounded. Furthermore, statistical variations of traffic intensity often lead to congestion that results in excessive delays and loss of packets, thereby significantly reducing the fidelity of real-time streams at their points of reception.

Efforts to define advanced services for both IP and ATM (Asynchronous Transfer Mode) networks have been conducted in two levels: (1) definition of service, and (2) specification of methods for providing different services to different packet streams. The former defines interfaces, data formats, and performance objectives. The latter specifies procedures for processing packets by hosts and switches/routers. The types of services defined for ATM include constant bit rate (CBR), variable bit rate (VBR) and available bit rate (ABR).

The methods for providing different services with packet switching fall under the general title of Quality of Service (QoS). The latest effort in QoS provision over the Internet is carried on by the Differentiated Services (DiffServ) Working Group of the Internet Engineering Task Force (IETF). DiffServ is working on providing QoS on a per-class basis, i.e., each switch provides a different service to packets belonging to different classes. The class to which a packet belongs is identified by a field in the IP packet's header. The DiffServ Working Group has re-defined the usage of the field originally called Type Of Service and has re-named the field DS (Differentiated Services) byte [K. Nichols, S. Blake, F. Baker, D. Black, "Definition of the Differentiated Services Field (D)S Field) in the IPv4 and IPv6 Headers," IETF Request for Comment RFC 2474, December 1998].

DiffServ relies on (i) a relatively small set of generic Per Hop Behavior (PHB), which define ways for individual switches to perform packet forwarding, and (ii) access control at the boundary of the network. A switch is configured to apply a specific PHB to each service class (i.e., switches are configured with a mapping between DS field value and corresponding PHB). A number of transport services can be built on those PHBs, including premium service, which is expected to deliver packets end-to-end within short delay and with low loss. One approach to an optical network that uses synchronization was introduced in the synchronous optical hypergraph [Y. Ofek, "The Topology, Algorithms And Analysis Of A Synchronous Optical Hypergraph Architecture", Ph.D. Dissertation, Electrical Engineering Department, University of Illinois at Urbana, Report No. UIUCDCS-R-87 1343, May 1987], which also relates to how to integrate packet telephony using synchronization [Y. Ofek, "Integration Of Voice Communication On A Synchronous Optical Hypergraph", IEEE INFOCOM'88, 1988]. In the synchronous optical hypergraph, the forwarding is performed over hyper-edges, which are passive optical stars. In [Li et al., "Pseudo-Isochronous Cell Switching In ATM Networks", IEEE INFOCOM'94, pp. 428-437, 1994; Li et al., "Time-Driven Priority: Flow Control For Real-Time Heterogeneous Internetworking", IEEE INFOCOM'96, 1996] the synchronous optical hypergraph idea was applied to networks with an arbitrary topology and with point-to point links. The two papers [Li et al., "Pseudo-Isochronous Cell Switching In ATM Networks", IEEE INFOCOM'94, pages 428-437, 1994; Li et al., "Time-Driven Priority: Flow Control For Real-Time Heterogeneous Internetworking", IEEE INFOCOM'96, 1996] provide an abstract (high level) description of what is called "RISC-like forwarding", in which a packet is forwarded, with little if any details, one hop every time frame in a manner similar to the execution of instructions in a Reduced Instruction Set Computer (RISC) machine.

Q-STM (Quasi-Synchronous Transfer Mode) [N. Kamiyama, C. Ohta, H. Tode, M. Yamamoto, H. Okada, "Quasi-STM Transmission Method Based on ATM Network," IEEE GLOBECOM'94, 1994, pages 1808-1814) uses a frame/subframe/slot structure to regulate the forwarding of ATM cells through the network. However, the authors do not suggest or mention the deployment of a common time reference, or the capability to transport variable size data packet, or the ability to combine "best effort" and variable bit rate (VBR) traffic types.

In U.S. Pat. No. 5,418,779 Yemini et al. disclose a switched network architecture with a time reference. The time reference is used in order to determine the time in which multiplicity of nodes can transmit simultaneously over one predefined routing tree to one destination. At every time instance the multiplicity of nodes are transmitting to a different single destination node. However, the patent does not teach or suggest the synchronization requirements among nodes, or the means in which it can be provided, or the method in which it can be used.

In the context of the Highball Project [D. L. Mills, C. G. Boncelet, J. G. Elias, P. A. Schragger, A. W. Jackson, A. Thyagarajan, "Final Report on the Highball Project," Technical Report 95-4-1, University of Delaware, April 1995] a network intended for a moderate number of users (10-100) was developed, deployed, and tested. Nodes are synchronized and transmission resources are reserved to flows so that packets always find output links available on every node traversed. No queuing is performed inside nodes; all queuing is done at the periphery of the network. This requires higher accuracy in the synchronization among nodes and affects the robustness of the system.

Architectures for data packet switching have been extensively studied and developed in the past three decades, see for example [A. G. Fraser, "Early Experiment with Asynchronous Time Division Networks", IEEE Networks, pp. 12-26, January 1993]. Several surveys of packet switching fabric architectures can be found in: [R. Y. Awdeh, H. T. Mouftah, "Survey of ATM Switch Architectures," Computer Networks and ISDN Systems, No. 27, 1995, pages 1567-1613; E. W. Zegura, "Architecture for ATM Switching Systems", IEEE Communications Magazine, February 1993, pages 28-37; A. Pattavina, "Non-blocking Architecture for ATM Switching", IEEE Communications Magazine, February 1993, pages 37-48; A. R. Jacob, "A Survey of Fast Packet Switches", Computer Communications Review, January 1990, pages 54-64].

Circuit switches exclusively use time for routing. A time period is divided into smaller time slices, each possibly containing one byte. The absolute position of each time slice within each time period determines where that particular byte is routed.

In accordance with one aspect of the present invention, time-based routing is supported with more complex periodicity in timing than circuit switching provides for. The time frames of the present invention delineate a vastly larger time period than the cycle time (i.e., the time slices) associated with circuit switching. The present invention also supports routing based on packet headers, which circuit switching cannot provide for.

Moreover, the present invention uses Common Time Reference (CTR). The CTR concept is not used in circuit switching (e.g., T1, T3, and the SONET circuit switching: OC-3, OC-12, OC-48, OC-192, and OC-768). Using or not using CTR has far reaching implications when comparing circuit switching and the current invention. For example, CTR ensures deterministic no slip of time slots or time frames, while enabling deterministic pipeline forwarding of time frames. This is in contrast to circuit switching, where (1) there are time slot slips, and (2) deterministic pipeline forwarding is not possible.

Several surveys of switching fabric architectures and interconnection networks can be found in: [G. Broomell, J. R. Heath, "Classification Categories and Historical Development of Switching fabric Topologies," Computing Surveys, Vol. 15, No. 2, June 1983; H. Ahmadi, W. E. Denzel, "A Survey of Modern High-Performance Switching Techniques," IEEE Journal on Selected Areas in Communications, Vol. 7, No. 7, September 1989; T. G. Robertazzi Editor, "Performance Evaluation of High Speed Switching Fabrics and Networks," IEEE Press, 1992; A. Pattavina, "Switching Theory", John Wiley & Sons, 1998].

Optical data communications include single wavelength standards, wherein a single data stream is transduced into a series of pulses of light carried by an optical fiber from source to destination. These pulses of light are generally of a uniform wavelength. This single wavelength vastly under-utilizes the capacity of the optical fiber, which may reasonably carry a large number of signals each at a unique wavelength. Due to the nature of propagation of light signals, the optical fiber can carry multiple wavelengths simultaneously with no degradation of signal, no interference, and no crosstalk imposed by the optical fiber. The process of carrying multiple discrete signals via separate wavelengths of light on the same optical fiber is known in the art as wavelength division multiplexing (WDM). Logically, wavelength division multiplexing may be thought of as equivalent to multiple single wavelength communications conducted in parallel, but the physical implementation does not require multiple optical fibers and therefore realizes cost savings.

The present invention permits a novel combination of time-based routing, which is similar but not identical to circuit switching, combined with data packet forwarding as in packet switching. This combination provides for communication of data via a reserved time frame mechanism, where time frames periods permit communications of a very large number of bytes that are scheduled and switched in a time-based fashion within reserved and scheduled time frames, while simultaneously providing for non-scheduled data packet (NSDP) traffic to be switched and routed via the same WDM (wavelength division multiplexing) optical channels. The non-scheduled data packet (NSDP) traffic can be transmitted during empty portions of an otherwise partially reserved and scheduled time frame period. The non-scheduled traffic can also be routed during fully reserved and scheduled time frame periods that have no scheduled traffic presently associated with them. Finally, NSDPs can be routed during unreserved time frames. The system can decode and be responsive to the control information in the non-scheduled data packet header.

There is a growing disparity between the data transfer speeds and throughput associated with the backbone or core of large networks, which may be in the range of one to tens of gigabits per second, and the data transfer speeds and throughput associated with end-user or node connections, which may be in the range of tens to hundreds of kilobits per second. Switching systems that function efficiently at the slow speeds required by end-user or node connections do not scale linearly or in a cost-effective manner to high speed and high performance variants. Existing circuit switches have additional problems as discussed above, in that with increasing data speeds comes a corresponding requirement for more accurate clocking.

Unlike a circuit switch that might potentially require switching a different route for each byte, the time frame switching in the present invention provides a novel mode of operation where the connection between an input port and an output port is only changed infrequently, such as on a time frame by time frame basis. This mode of operation is an enabling technology to utilize purely optical switching apparatus, as it circumvents the problems typically associated with long switching cycle time.

Moreover, the present invention enables the utilization of very simple interconnection networks such as Banyan Networks [L. R. Goke, G. J. Lipovski, "Banyan Networks for Partitioning Multiprocessor Systems," 1st Annual Symposium on Computer Architecture, December 1973, pages 21-28] whose utilization in other systems may not be advisable due to their blocking features.

The Dynamic Burst Transfer Time-Slot-Base Network (DBTN) [K. Shiomoto, N. Yamanaka, "Dynamic Burst Transfer Time-Slot-Base Network," IEEE Communications Magazine, October 1999, pages 88-96] is based on circuit switching. A circuit is created on-the-fly when the first packet of a burst is presented to the network; the first and subsequent packets are transported through the network over such circuit.

Dynarc and Net Insight, two Sweden based companies, commercialize switches for Metropolitan Area Networks (MANs) based on Dynamic synchronous Transfer Mode (DTM) [C. Bohm, P. Lindgren, L. Ramfelt, P. Sjodin, "The DTM Gigabit Network," Journal of High Speed Networks, Vol. 3, No. 2, 1994. C. Bohm, M. Hidell, P. Lindgren, L. Ramfelt, P. Sjodin, "Fast Circuit Switching for the Next Generation of High Performance Networks," IEEE Journal on Selected Areas in Communications, Vol. 14, No. 2, pages 298-305, February 1996.] DTM deploys a structure of frames and small slots (64 bits) to perform resource allocation and circuit switching. Slots are allocated to the end-systems according to a predefined distribution; a distributed algorithm based on the deployment of control slots is used to reallocate unused slots.

SUMMARY OF THE INVENTION

In accordance with the present invention, a fast switching method is disclosed and is tailored to operate responsive to a global common time such that the switching delay from input to output is known in advance and is minimized in a deterministic way. Consequently, such a switch can be employed in the construction of a backbone network using optical fibers with dense wavelength division multiplexing (DWDM). Such optical fiber links have a transmission rate, with multiple wavelengths, of a few terabits (1012) per second.

The design method disclosed in this invention minimizes the time required for the routing decision and switching of every data packet. Consequently, for a given solid state technology, memory access time and memory word width, this method can support the highest speed optical DWDM links. Moreover, the above is independent of the number of switch ports.

The switching and data packet forwarding method combines the advantages of both circuit and packet switching. It provides for allocation and exclusive use of transmission capacity for predefined connections and for those connections it guarantees loss free transport with low delay and jitter. When predefined connections do not use their allocated resources, other non-reserved data packets can use them without affecting the performance of the predefined connections.

Under the aforementioned prior art methods for providing packet switching services, switches and routers operate asynchronously. The present invention provides real-time services by synchronous methods that utilize a time reference that is common to the switches and possibly end stations comprising a wide area network. The common time reference can be realized by using UTC (Coordinated Universal Time), which is globally available via, for example, GPS (Global Positioning System--see, for example: [Peter H. Dana, "Global Positioning System (GPS) Time Dissemination for Real-Time Applications", Real-Time Systems, 12, pp. 9-40, 1997]. By international agreement, UTC is the same all over the world. UTC is the scientific name for what is commonly called GMT (Greenwich Mean Time), the time at the 0 (root) line of longitude at Greenwich, England. In 1967, an international agreement established the length of a second as the duration of 9,192,631,770 oscillations of the cesium atom. The adoption of the atomic second led to the coordination of clocks around the world and the establishment of UTC in 1972. The Time and Frequency Division of the National Institute of Standards and Technologies (NIST) (see http://www.boulder.nist.gov/timefreq) is responsible for coordinating UTC with the International Bureau of Weights and Measures (BIPM) in Paris.

UTC timing is readily available to individual PCs through GPS cards. For example, TrueTime, Inc. (Santa Rosa, Calif.) offers a product under the trade name PCI-SG, which provides precise time, with zero latency, to computers that have PCI extension slots. Another way by which UTC can be provided over a network is by using the Network Time Protocol (NTP) [D. Mills, "Network Time Protocol" (version 3) IETF RFC 1305]. However, the clock accuracy of NTP is not adequate for inter-switch coordination, on which this invention is based.

In accordance with the present invention, the synchronization requirements are independent of the physical link transmission speed, while in circuit switching the synchronization becomes more and more difficult as the link speed increases. In accordance with the present invention, routing is not performed only based on timing information: routing can be based also on information contained in the header of data packets. For example, Internet routing can be done using IP addresses or using an IP tag/label when MPLS is deployed.

One embodiment of the present invention utilizes an alignment feature within an input port for aligning incoming data packets to a time frame boundary prior to entry to a switching fabric. This embodiment has the additional benefit of providing for filtering non-reserved traffic from the data packet stream and routing said traffic to a separate routing controller for best effort transport. The system decodes and is responsive to control information in the non-reserved data packet header. The remainder of the traffic represents reserved traffic that is first aligned to a time frame boundary and then routed through the switch fabric on a subsequent time frame, thus preserving the synchronous operation of the system. The present invention also provides means to reintegrate the filtered non-scheduled traffic into idle portions as may coexist within the scheduled traffic streams.

One embodiment of the present invention utilizes a deferred alignment feature, which permits the alignment of incoming data packets to be deferred after preliminary routing and queuing has been performed. This embodiment trades additional storage required for a larger plurality of queues for reduced complexity required in the switch fabric. The switch fabric becomes simpler because it is logically divided into a first portion and a second portion, the first portion of which can be relocated upstream of (i.e., before) the alignment buffer queues. By relocating the first portion to a position before the alignment buffer queues, the first portion of the switch fabric may be implemented as a simple data path expander to fan out the data to a large plurality of queues. The complexity and throughput requirements of each queue are also reduced as the functionality is spread out over a wider number of queues.

A novel control mode is provided by the present invention where a packet header comprises new in-band signal information to establish, maintain, and dis-establish (or destroy) a reserved traffic channel. The system decodes and is responsive to the control information in the data packet header. In this control mode, a specially designated data packet works as a "trailblazer" by signaling to each switch in a plurality of connected switches that it is the first of an expected train of associated data packets. The switches of the present invention respond if able by establishing a reserved data channel, a reserved transfer bandwidth, or by reserving capacity for the traffic associated with and following the specially designated data packet. In an analogous fashion, a terminating data packet signals to each switch in a plurality of connected switches that it is the last of a group or train of associated data packets. The switches of the present invention respond by destroying, reallocating, or reclaiming the data transfer capacity or bandwidth that had been made available to the train of data packets. Interstitial data packets within a train of data packets are marked as such to permit the switches to quickly and easily identify the data packet as one belonging to a scheduled and reserved train of data packets and to the corresponding reserved bandwidth or capacity. Data packets not having the special designations indicated above are treated in the conventional way, where they are generally but not exclusively carried on a best effort basis. Note that the in-band scheduling and reservation of the present novel control mode is independent of but operates concurrently and in cooperation with any other reserved traffic mechanism implemented in the switching systems.

A novel time frame switching fabric control is provided in accordance with an alternate embodiment of the present invention, which stores a predefined sequence of switch fabric configurations, responsive to a high level controller that coordinates multiple switching systems, and applies the stored predefined sequence of switch fabric configurations on a cyclical basis having complex periodicity. The application of the stored predefined switch fabric configurations permits the switches of the present invention to relay data over predefined, scheduled, and/or reserved data channels without the computational overhead of computing those schedules ad infinitum within each switch. This frees the switch computation unit to operate relatively autonomously to handle transient requests for local traffic reservation requests without changing the predefined switch fabric configurations at large, wherein the switch computation unit provides for finding routes for such transient requests by determining how to utilize underused switch bandwidth (i.e., "holes" in the predefined usage). The computational requirements of determining a small incremental change to a switch fabric are much less than having to re-compute the entire switch fabric configuration. Further, the bookkeeping operations associated with the incremental changes are significantly less time-consuming to track than tracking the entire state of the switch fabric as it changes over time.

These and other aspects and attributes of the present invention will be discussed with reference to the following drawings and accompanying specification.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of one embodiment of a switch connected to a plurality of WDM links with a switch scheduler in accordance with the present invention;

FIG. 2 is a timing diagram of a common time reference (CIR) that is aligned to the coordinated universal time (UTC) standard, as utilized by the present invention, wherein the CTR is divided into a plurality of contiguous periodic super-cycles each comprised of at least one contiguous time cycle each comprised of at least one contiguous time frame, wherein the super-cycle is equal to and aligned with the UTC second;

FIG. 3 is a schematic block diagram of a virtual pipe and its timing relationship with a common time reference (CTR) as in the present invention;

FIG. 4 illustrates the mapping of time frames into and out of a node on a virtual pipe of the present invention;

FIG. 5A is a schematic block diagram illustrating at least one serial transmitter and at least one serial receiver connected with a WDM link, in accordance with the present invention;

FIG. 5B is a table illustrating a 4B/5B encoding scheme for data;

FIG. 5C is a table illustrating a 4B/5B encoding scheme for control signals;

FIG. 6A is a map of a data packet with a header, as utilized in accordance with the present invention;

FIG. 6B illustrates a mapping of additional details of the encoding of the data packet of FIG. 6A;

FIG. 7 is a schematic block diagram of an input port in accordance with the present invention;

FIG. 8 is a flow diagram illustrating the operation of the routing controller in accordance with the present invention;

FIG. 9 is a schematic block diagram of an embodiment of a packet scheduling controller in accordance with the present invention;

FIG. 10 is a schematic block diagram of an alternate embodiment of a packet scheduling controller in accordance with the present invention;

FIG. 11 is a flow diagram describing the operation of the packet scheduling and rescheduling controllers of FIGS. 9 and 10;

FIG. 12 illustrates details of the input request, input reject, and input schedule messages in accordance with the present invention;

FIG. 13 is a flow diagram illustrating the operation of the select buffer and congestion controllers of FIGS. 9 and 10;

FIG. 14 illustrates the four pipelined forwarding phases of forwarding data packets in accordance with the present invention;

FIG. 15 is a schematic block diagram of a four pipelined forwarding phases of forwarding data packets in accordance with the present invention;

FIG. 16 is a schematic block diagram of one embodiment of the switching fabric, with its fabric controller, in accordance with the present invention;

FIG. 17 is a schematic block diagram of an output port in accordance with the present invention;

FIG. 18 is a flow diagram illustrating the operation of a pipelined forwarding phase of the output port of FIG. 17;

FIG. 19 is a flow diagram illustrating the operation of another pipelined forwarding phase of the output port of FIG. 17;

FIG. 20 is a flow diagram illustrating the operation of the switch scheduling controller of FIG. 1;

FIG. 21 illustrates details of the scheduling computation of the switch scheduling controller in accordance with the present invention;

FIG. 22 illustrates additional details of the scheduling computation of the switch scheduling controller in accordance with the present invention;

FIG. 23 illustrates further details of the scheduling computation of the switch scheduling controller in accordance with the present invention;

FIG. 24A is a functional diagram of a switch with the Fast Switching mode of operation, which implies that there are pre-computed schedules for transferring the incoming data packets to their respective output ports;

FIG. 24B is a timing diagram of three pipelined forwarding phases, with predefined schedules for forwarding data packets in accordance with the present invention;

FIG. 25 provides an example of a fabric controller that uses a plurality of Fast switching matrices, where there is a different switching matrix for a subset of time slots in every time frame, for each time frame in every time cycle, and for each time cycle in every super-cycle in accordance with the present invention;

FIG. 26 illustrates a wave division multiplexing (WDM) switch that is connected to optical link with multiple wavelengths, wherein each of the wavelengths constitutes a communication channel that has a time division multiplexing (TDM) structure with time frames, time cycles and super-cycles in accordance with the present invention;

FIG. 27 illustrates multi-dimensional mapping with four input variables as an example: p-in--input port #, w-in--input wavelength (color), t-in--time frame # in (within a time cycle), c-in--time cycle # in (within a super-cycle); and four output variables: p-out--output port #, w-out--output wavelength (color), t-out--time frame # out (within a time cycle), c-out--time cycle # out (within a super-cycle) in accordance with the present invention;

FIG. 28 illustrates an example of pipeline forwarding of time frames, in accordance with the present invention;

FIG. 29 illustrates an example of mapping time frames, received over the same wavelength received through multiple input ports, to one wavelength (channels) on the same output port, in accordance with the present invention;

FIG. 30 illustrates an example of multi-dimensional mapping for all time-driven optical switching with no wavelength conversion, the optical switching being responsive to the common time reference in accordance with the present invention;

FIG. 31A is a schematic diagram of an all optical switch with at least one optical switching fabric, which switches a plurality of optical wavelengths, wherein the optical switching matrix (as in FIG. 30, for example) changes every time frame;

FIG. 31B is a timing diagram of the all optical switch operation with two phases: one in which the actual switching is performed and the other in which the current switching matrix is being replaced by a new switching matrix;

FIG. 32A is a schematic diagram of a multiple fabric switch;

FIG. 32B is a timing diagram of a switching operation that is responsive to the common time reference 002 with three pipeline forwarding phases that enable the operation with the pre-computed schedules with the Fast Queuing Method;

FIG. 33A is a functional description of a switch with 16 ports--each with 16-wavelength division multiplexing optical channels, such that it is possible to transfer: From (any time frame (TF) of any Channel at any Input) To (a predefined time frame (TF) of any Channel at any Output);

FIG. 33B is a timing diagram of a switching operation that is responsive to the common time reference 002 with two pipeline forwarding phases;

FIG. 34 is a functional block diagram illustrating a wavelength division multiplexing input port with a plurality of serial receivers, serial-to-parallel conversion and a plurality of alignment subsystems;

FIG. 35 is a functional block diagram of the alignment subsystem that operates responsive to CTR and the serial link relative timing;

FIG. 36 is a timing diagram of the alignment subsystem operation responsive to CTR and the serial link relative timing;

FIG. 37 is a block diagram and schematic of the structure of a switch and a fabric controller with memory for a plurality of switching matrices;

FIG. 38 is illustrates a wavelength division multiplexing output port;

FIG. 39 is a functional block diagram of a wavelength division multiplexing input port with data packet filters for detecting non-scheduled data packets, which are forwarded to a routing module;

FIG. 40 is a block diagram of a routing module;

FIG. 41 is a block diagram of a data packet filter connected to an alignment subsystem that is connected to a switch fabric and a fabric controller;

FIG. 42 is a block diagram of a switch design with a 16-to-256 expander, wherein the expander output lines are connected to alignment subsystems;

FIG. 43 is a more detailed description of the 16-to-256 expander of FIG. 42;

FIG. 44 is a functional block diagram of the connection from the alignment subsystems to an output port via a plurality of selectors;

FIG. 45 is a functional block diagram of an SVP (synchronous virtual pipe) interface with per time frame queues;

FIG. 46A is a functional block diagram of an SVP interface with per SVP queues;

FIG. 46B is a functional block diagram of multiple SVP interfaces to a multi-protocol time driven SVP switch;

FIG. 47 is a system block diagram of a network with a plurality of multi-protocol time driven SVP switches that are connected to SVP interfaces and other vendors' optical 110 cross connects (OXCs), showing channels, interfaces, and so forth;

FIG. 48 is a high level diagram of communications layering and a description of a two layer system, wherein the low/inside layer is dense wavelength division multiplexing (DWDM) and the outer layer is IP/MPLS;

FIG. 49A is a straight connection of a 2-by-2 switching block;

FIG. 49B is a cross connection of a 2-by-2 switching block;

FIG. 49C is a diagram of an 8-by-8 multi-stage interconnection switch that is constructed of 2-by-2 switching elements;

FIG. 50A is a comparison table of a multi-stage interconnection switch with a crossbar switch; and

FIG. 50B is a block diagram of a 256-by-256 multi-stage interconnection switch that is constructed of 4-by-4 switching elements.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

While this invention is susceptible of embodiment in many different forms, there is shown in the drawing, and will be described herein in detail, specific embodiments thereof with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to limit the invention to the specific embodiments illustrated.

The present invention relates to a system and method for switching and forwarding data packets over a packet switching network with optical WDM (wavelength division multiplexing) links. The switches of the network maintain a common time reference (CTR), which is obtained either from an external source (such as GPS--Global Positioning System) or is generated and distributed internally. The common time reference is used to define time intervals, which include super-cycles, time cycles, time frames, time slots, and other kinds of time intervals. The time intervals are arranged both in simple periodicity and complex periodicity (like seconds and minutes of a clock).

A packet that arrives to an input port of a switch, is switched to an output port based on either specific routing information in the packet's header (e.g., IPv4 destination address in the Internet, VCI/VPI labels in ATM, MPLS-multi-protocol label switching-labels) or arrival time information. Each switch along a route from a source to a destination forwards packets in periodic time intervals that are predefined using the common time reference.

A time interval duration can be longer than the time duration required for communicating a data packet, in which case the exact position of a data packet in the time interval is not predetermined. A data packet is defined to be located within the time interval which contains the communication of the first bit of the packet, even if the length of the packet is sufficiently long to require multiple time intervals to communicate the entire data packet.

Data packets that are forwarded inside the network over the same route and in the same periodic time intervals constitute a virtual pipe and share the same pipe-ID or PID. A pipe-ID or PID can be either explicit, such as a tag or a label that is generated inside the network, or implicit such as a group of IP addresses or the combination of fields in the data packet header. A virtual pipe can be used to transport data packets from multiple sources and to multiple destinations. The time interval in which a switch forwards a specific packet is determined by the time it reaches the switch, the current value of the common time reference, and possibly the packet's pipe-ID.

A virtual pipe can provide deterministic quality of service guarantees. In accordance with the present invention, congestion-free packet switching is provided for pipe-IDs in which capacity in their corresponding forwarding links and time intervals is reserved in advance. Furthermore, packets that are transferred over a virtual pipe reach their destination in predefined time intervals, which guarantees that the delay jitter is smaller than or equal to one time interval.

Packets that are forwarded from one source to multiple destinations share the same pipe-ID and the links and time intervals on which they are forwarded comprise a virtual tree. This facilitates congestion-free forwarding from one input port to multiple output ports, and consequently, from one source to a multiplicity of destinations. Packets that are destined to multiple destinations reach all of their destinations in predefined time intervals and with delay jitter that is no larger than one time interval.

A system is provided for managing data transfer of data packets from a source to a des