Network Architecture: Why Choices Must Be Made
Most if not all network inputs are finite:
The speed of light is finite
Resources are finite
Externalities are finite
As inputs are finite, choices have to be made. Which resources will be used, in what quantities, for what purposes.
Figure 1. What drives the need to make choices
If Resources Were Infinite
A network control plane could transmit, store, and process any amount of state (information)
There would be no scale-drivers for using address aggregation/summarization
There would be no scale-drivers for dividing networks into areas
There would be no capacity-drivers for Quality of Service, ECMP, or Traffic engineering
A network could power and cool any amount of equipment, at any load
Some amount of policy could be eliminated
Many resource saving approaches and mechanisms would not be required
The industry discussion about how much state is in control planes, the occasional challenge with providing power / cooling for large data centers, the presence of bandwidth management techniques such as QoS, ECMP, and TE, numerous software optimizations, and some portion of policy, are manifestations of the finite nature of resources. There is a direct relationship between control/forwarding plane load and the extent to which networking approaches/techniques such as these are used. That relationship may vary in different parts of a network.
Many of the approaches and mechanisms in network/equipment design, drive, in part, complexity in networks. There are other drivers as well. It is not the intent of this article to cover all the drivers of complexity.
If resources were infinite, networks would be simpler, yet less efficient, because efficiency would not be a concern. Often in economic systems we find that moments of seemingly infinite resources and/or lack of competition for resources and customers, leads to inefficiencies that become problematic when the period of infinite resources and lack of competition passes. Periods of seemingly infinite resources are often not sustainable / are short-lived either due to direct scarcity of resources or externalities. Periods of seemingly infinite resources should be a strategic concern in terms of inefficiencies that aggregate during them.
The Speed of Light is Finite
Information is transmitted in information networks using electromagnetic waves. Commonly light in a fiber optical cable and radio waves.
The speed of electromagnetic waves is not infinite, it is finite, constant in a vacuum, regardless of inertial frame. As a result, information cannot be everywhere, all at the same time. There is always propagation delay between sender and receiver, there is always information relativity (differences in information, and sequence in which information is received). As a consequence:
No point of space, person, or thing, is omniscient (all knowing)
There are differences of opinion about the sequence of events
No distributed router has perfect knowledge of the rest of the network and sequence of events.
No centralized controller has perfect knowledge of the rest of the network and sequence of events
Networks should be designed and operated assuming information is not perfect
Resources are finite
Over the long run, resources are finite. Due to financial/economic dynamics and externalities, there is a limit to how much compute, storage, network, energy, cooling, space, and people, that can be deployed by a network operator.
High growth companies can sometimes act as if they are immune to the laws of economics; as if resources are infinite. What tends to happen for such companies, is that a resource not generally considered, for example energy, spikes to levels that are not planned for, and that resource becomes scarce. Another potential issue is externalities feed back into the resource supply chain in negative ways - see next section.
Finite network resources include:
Information
Compute
Storage
Links
Switching
Energy
Cooling
Space
People
Externalities are Finite
There is a history and experience of entities causing externalities, not directly paying for them. There may be significant time gaps between when an entity receives the positive benefit from creating a negative externality and the negative feedback from continuing to do so. Externalities can form a negative feedback to an entity when there is:
Impact to supply of another resource.
Violation of the entity's own values.
Violation of the entity's customer’s values.
Protest by interest groups.
Public sector intervention.
It may take time for externalities to manifest as negative feedback to the entity creating them, but over the long run, there is often feedback. The formation of negative feedback is not guaranteed, but if it is impacting some other segment of society, in a compellingly negative way, for long enough, at an accelerating rate, it generally will. In the worse case scenario, destruction of the environment in which the entity operates could threaten the supply of many resources that are inputs to the entity.
Consumers are increasingly becoming values based buyers, but they still require the information needed to determine whether an entity they are purchasing goods and services from is acting in a way aligned with the consumer's values. The creation and awareness of necessary information about externalities can take time and its own resources.
Conclusion
The finite nature of resources (the ability to create and/or afford) and the unlikelihood of significant negative externalities persisting over the long run, means that choices have to be made about the quantity, quality, and usage of resources. Choices also have to be made about what capabilities must be supported and enabled in a network. Capabilities including ECMP, QoS, Traffic Engineering, address summarization, network areas, some aspects of policy, and importantly, state design and management in the control plane.