A. What is O-IP?
The
basics of an O-IP system are to allow the use of Internet Protocol (IP)
over narrow band systems with all the benefits of a licensed RF path.
The data rates will be in the 4800 to 19200 bps range with a higher
effective throughput. The O-IP product must be able to manage the
Ethernet and IP packets such that only a minimum required amount of
overheard information is sent through the air. The final O-IP product
will manage both the amount of packet overhead sent over the air on the
RF link and will also apply data compression algorithms to reduce the
amount of user data sent.
B. Why an Optimized Internet Protocol Device?
Why
is there a need for Optimized Internet Protocol (O-IP) communications?
The Supervisory Control and Data Acquisition (SCADA) industry is moving
toward the Internet Protocol (IP) enabled network in a very determined
manner. There are several reasons: the need for network manageability;
the movement of manufacturers to IP based products, the general movement
away from serial connections and the fact that many SCADA systems and
automation groups have been moved into existing Networking control
groups or Information Technology (IT) organizations.
Greater
distance Radio Frequency (RF) paths are achieved with narrow band
Frequency Modulated (FM) licensed products. Since the frequencies are
licensed and regulated, power amplifiers and specialized RF filtering
products can be used to give system reliable spans measured in tens of
miles, not just miles. It is not atypical for a narrow band Ultra High
Frequency (UHF) SCADA system to cover 50 or 75 miles of territory with
no repeaters or single systems. Some Very High Frequency (VHF) based
systems reach in excess of 90 miles as a routine design requirement. The
fact that the frequencies are assigned by a governing agency (Federal
Communications Commission) and coordinated by local frequency
coordinators also give a certain level of certainty that interference
will be less likely and there is some recourse should it occur. This is
not necessarily a feature of typical wide-band unlicensed products. The
FCC Part 15 devices (spread spectrum) are required to "co-exist" with
any interference and it is not uncommon that a move to a licensed
frequency alleviates interference problems.
The
movement away from RS-232 serial communications methods poses
challenges. There is a significant installed base of serial-based
Integra communications systems working on narrow band (25 kHz and 12.5
kHz channels). These systems are typically slow to mid-speed (1200-19200
bits per second (bps)) applications. It was not too long ago 9600 or
19200 bps was considered very fast in the SCADA business! There is also a
large installed base of serial based Integra spread spectrum products.
In either case, the wholesale replacement in terms of cost, downtime and
staff time is appreciable and they make alternatives worth looking at.
C. How Will O-IP Work?
A
typical Ethernet message consists of a lot of overhead information to
make sure the data arrive at their intended destination. However, if the
design of the network is known, a certain amount of that header
information can be limited, lowering the on-air traffic.
Typical Ethernet User Datagram Protocol (UDP) or Transmission Control Protocol /IP (TCP/IP) Overhead:
In
many cases the overhead can exceed the actual SCADA message, i.e., a
54-byte header to send a 6-byte SCADA message. This would not be an
acceptable or efficient method of SCADA communications.
Dataradio's
mobile VIS (Vehicular Information System) optimized IP product has been
in service for sometime now. It has been deployed in many locations
with strong success. Taking lessons from that product development,
Dataradio Engineering developed a SCADA Optimized IP solution that
focuses on the particular needs of the SCADA user for IP connectivity.
The
requirement for duplicate packets generated by TCP/IP are significantly
reduced. Customized Data Compression algorithms afford up to a 50%
compression rate for data, dependent on the data type. Header reduction
is a fixed reduction of 25%.
This
type of network intelligence is designed into a small microprocessor
board that will be available as an add-on enclosure (Phase One) and an
integral (Phase Two) with Dataradio products. There will not be a need
for a separate personal computer or server in the system. Set up will be
via personal computer and a table file structure and/or command
line/HTTP based interface.
When
there is high bandwidth/short distance available, a Media Access
Control (MAC) layer bridge with little or no filtering may work well.
Inefficiencies in data transmission are compensated for with the higher
speed of such a link. However, if a similar approach is taken over a
narrow band FM RF link, performance will not be sufficient to allow
acceptable operation. This is where the Optimized IP connection
methodology is best utilized, allowing a reasonable connection in these
cases.
Remote Terminal Unit (RTU) Test Set-up:
Figures
1 and 2 are diagrams that outline two test set-ups that were used to
verify and test the operation of the O-IP device. Test set-ups were
based on user feedback as to the type of possible networks. Other
connections are likely however these two test scenarios represent how we
would expect the product to be put into service on an initial basis.
Additional addressing data is provided to indicate the set-up format.
Figure 1: Test RTU Network Setup:
Figure 2: IP Native RTU and Terminal Server Network
D. What Are Some System Design Considerations of an O-IP System?
System
design criteria requires some up-front work, especially since there are
not unlimited speed and bandwidth allocations. SCADA system design is
not foreign to SCADA users, however, with Local Area Network (LAN)
systems a larger amount of the system "design" is left to the equipment
and less than optimal designs can be compensated for by the high
throughput enjoyed in LAN type systems. Some design criteria are listed
below:
-
These
SCADA O-IP systems will not support web surfing. Email systems such
as Outlook and Lotus Notes will not be efficient because of the
half-duplex nature of the radio channel and full-duplex nature of
TCP. The overhead is simply too large and the system responsiveness
would likely not be acceptable. A simple text based email system
would work if not overused. Drive sharing and other common network
components will not function well.
-
Efficient
data throughput is based on SCADA oriented messaging size.
Structures of the SCADA messaging need to be understood and perhaps
adjusted to fit the application. Throughput is based on application
architecture; i.e., half-duplex or full-duplex, number of devices
supported and message size. This is in effect no different than what
is currently done for serial based systems.
-
Rockwell
Automation offers the following advice: "The recommended
Ethernet/IP network topology for control applications is an active
star topology (10 MBPS and 100 MBPS Ethernet can be mixed) in which
groups of devices are point to point connected to a switch. The
switch is the heart of the network system." O-IP is closer to a WAN
environment, an Ethernet switch (star topology) is used for
deterministic networks and deterministic response times while a WAN
tends to be designed for more flexible approach to data movement.
The O-IP environment allows for the chance of a data collision
unless a polling-based application is used - this is a more typical
SCADA application. In this type of optimized system, the routing and
gateway capabilities of O-IP are utilized to better manage on-air
RF traffic and maintain system reliability - we need to work smarter
not merely faster.
-
Dynamic
Host Configuration Protocol (DHCP) will not be supported in the
initial offering. Design requirements should limit any application
protocol based on IP broadcasting. We recommend using multicasting
instead. There has not been a strong requirement indicated for this
feature which can create significant overhead. The system has to be
laid out with as much determinism as possible. If elements are
changed, then the tables get changed. Typically SCADA systems have
minimal change so change control can be implemented and table
up-dates managed. Simply stated, SCADA systems are typically static
address based.
-
The
O-IP product will function as a gateway and router intelligently
limiting the amount of traffic it forwards on to the RF network. As a
comparison, MAC layer bridging would forward all broadcast messages
generated on local LAN; i.e., IP broadcast, Internet Packet
exchange (IPX) broadcast would forward Address Resolution Protocol
(ARP) requests over the RF channel.
-
There
is no limit on the number of Remote Terminal Units
(RTU)/Programmable Logic Controllers (PLC) but network latency is
dependent upon the number of RTU/PLCs on the network. Most serial
systems require some kind of traffic calculation/review to determine
how many sites can be polled and respond within a given time frame.
Most network administrators and vendors have tools that assist in
calculating the system latency, throughput and scan rates. Dataradio
provides at least two types for general rule-of-thumb use. System
designers may need to work with system programmers to understand data
structures and required throughput rates for the application. This may
also involve the process control/system engineers to understand
what overall system performance criteria are. It has been the
experience of Dataradio Technical Services that when these items are
not addressed, system performance is not optimal either serial
communications or LAN. There are networking tools available to
assist in system performance evaluation and some allow for system
performance extrapolation. Parameters such as tuning of TCP/IP
parameters (Maximum Transmission Unit (MTU) size, MSS size etc.)
will need to be set correctly. Dataradio will publish starting
benchmarks for these parameters as work progresses with more systems
and products.
-
How
will the SCADA network be linked to any other corporate networks -
through hubs or switches? How will the demands for non-SCADA
information be handled? Tight control needs to be exercised or
random data requests could easily impact the basic system
performance. Requests addressed to RTUs/PLCs/Intelligent Electrical
Devices (IED) will be passed on but if those requests come from a
non-SCADA application (Engineering, Accounting, and Maintenance) the
amount of traffic can impact system performance. Understanding how
broadcast messages move through the system is important. O-IP will
have the capability to enable or disable broadcast IP messages in
the O-IP set-up. Limiting the number of broadcasts will keep traffic
levels down as well.
-
System
addressing needs to be thought out in advance to avoid duplicate
addresses and use of illegal addresses. If the SCADA networks are
kept isolated from other networks private IP addresses can be used
for RTU/PLCs.
-
What
types of devices will be on the network? RTUs, PLCs, IEDs, terminal
servers, meters and other process control devices (virtually any
device that uses IP as a network layer) can be used with O-IP. Each
type of device has a communication profile that needs to be taken
into account as far as messaging size, latency control, reply
message size and ad-hoc messaging. Network dynamic control is a part
of future Dataradio O-IP work.
If the system is a class C network, up to 254 devices could be on
the segment. But having a device count capability is not the same as
having the throughput capability. If all the messaging is small and
short, 254 devices could easily be supported. What it really gets
down to is this: The more points there are to monitor, the longer it
will take the system to poll them. Network latencies will impose
longer scan times on data collection routines.
-
What
protocols can be used with an O-IP system? Protocols such as UDP,
TCP, Internet Control Message Protocol (ICMP), ARP, Modbus/IP (IP
and a Modbus header), Modbus/TCP, ASCII over IP, Distributed Network
Protocol (DNP) 3.0 are supported (timing constraint issues have
come up with DNP 3.0 in any number of applications- not just O-IP.
Review of the application and latencies is necessary.
A. Items that should be reviewed are:
-
What is a typical data request size?
-
What is the typical data reply payload size?
-
What latencies are allowed by the PLC/IED/RTU?
-
Will
LAN system latencies work with RF system latencies? (The
longest latency will govern the system performance).
-
A
review of timing requirements for the SCADA host program needs
to include timing for message turn-around, message reply timer,
total message timer, and other system timers.
-
Does
the design of the network and other network devices allow for
longer latencies inherent in an RF system? Some devices
internally buffer data to avoid latency time issues; others
allow a longer latency.
-
Once
network design issues are addressed, full system design can be
completed and implementation can go forward. Progressive system
testing should be performed so that issues can be addressed and
resolved in smaller groups as opposed to turning the entire system
on and then trying to "whittle down" issue areas.
-
Most
end users tend to use a few protocols, devices and designs. Once
this effort is done for the first system, a lot of the information
will be able to be transferable for use in other systems. These
elements are also part of any design effort for maximized system
operation. These efforts are often the difference between a
marginally operating and a truly efficient system.
E. Conclusion:
O-IP
has a place in the RF market, especially supporting the narrow band FM
sector. It represents a significant step forward allowing a greater
connectivity option for those users who are distance constrained and
want to use their legacy Integra installations. It also provides a
migration path that will minimize the cost of conversion to a more
manageable level.
Used
in conjunction with the Integra wireless modem, the full feature set of
the Integra system is available to the user. This includes online,
offline and remote diagnostics, plus Dataradio infrastructure products,
base stations, repeaters, rack mounting, power supplies, power
amplifiers, antenna kits, National Electrical Manufacturers Association
(NEMA) enclosures and High Availability (redundant bases and repeaters)
options. The High Availability option allows for a "no single point of
failure" system-back up capability for those critical links that need
guaranteed uptime.
The
product will be available initially as an add-on product, allowing for
maximum up-grade flexibility. However, the end user will need to do some
up-front work to take as full advantage of the capabilities. In many
cases this information should (generally) be available as normal system
design or maintenance information. The end user has the responsibility
of managing the network for maximum performance, understanding that O-IP
is not a panacea for all IP network needs but a targeted answer for
certain needs.
Notes
-
All respective trade names trademark, copyrights, and service marks are property of their respective owners.
-
The
use of a trade name or product name does not necessarily constitute
an endorsement of that product, device, or software.
This
article was written and provided by Harry Ebbeson, Manager of Technical
Services at Dataradio COR Ltd. Dataradio is a leading designer and
manufacturer of advanced wireless data products and systems for mission
critical applications.
Source:-http://www.automation.com/library/articles-white-papers/hmi-and-scada-software-technologies/optimized-internet-protocol-network-for-scada-systems
