Not too long ago, a fully automated production environment was designed for an
Agilent facility. It permits so-called 24/7 (24 h/d, seven d/wk) testing—without human intervention. The goal was to lower
manufacturing overhead costs and to increase production capacity.
In principle, achieving this was straightforward. But a number of unforeseen problems associated with automation conspired to reduce
performance to an unacceptable level. Fortunately, statistical process control (SPC) came to the rescue. SPC was used to identify problems,
permitting almost continuous station operation with close to 100 percent process yield, as well as traceability to national standards.
Let's go back in time and trace the events leading to the adoption of SPC. Our story begins about two years ago with equipment manufactured
on production lines within Agilent's RF Communications division at Queensferry, Scotland. This equipment was tested against specifications by
processes comprising a number of function-specific systems operated by engineers.
These engineers were responsible for "walking a product through" each step of the production test process. The goal was to ensure that the
appropriate test schedule was followed for the product, as well as managing various on-line demands from the test systems.
This was the model of production test procedure for many years at the Queensferry facility, and although increased product complexity has
driven greater test capability, the implementation remained largely unchanged.
An Inadequate Model
Back to top
With increasing demands placed on the facility's production department, the existing model was inadequate to meet targets for production yield
and capacity.
A number of factors were limiting production performance. These included unrepeatable measurements, erroneous results (no faults found), manual
processes, and test system calibration frequency. In addition, manufacturing costs associated with capital investment in test hardware and the
overhead cost of production personnel were becoming unsustainable. Something had to be done.
BEFORE…
A typical production test environment!
AFTER…
The Yellowstone environment.
A decision was made to change to nearly complete automation, with the task of testing a product handled by a robotic system. This reasonable-cost
system, code-named Yellowstone (not to be confused with Rambus memory devices), proved successful at Agilent for products of a simple nature.
Six Advantages
Back to top
There were six main advantages for operating Yellowstone. First, Yellowstone would ensure maximum utilization of assets through continuous production,
increasing return on invested capital. Next, it would ensure a reduction in manufacturing overhead through fewer production-test operators.
Yellowstone would also permit the redeployment of skilled test engineering staff so they could do more value-added work, such as instrument rework and
repair.
Yellowstone would also provide a manufacturing test process that would be consistent and repeatable. It would also ease production congestion and
bottlenecks by means of defined automation rules. Lastly, it would give greater control of processes, leading to a more predictable output.
Let's now follow the evolution of a production test system, tracing the change from a legacy-type ATE system to a fully automated robotic one.
The example uses an Agilent ET42803 Power Test Station. It's used to verify the accuracy of CW (continuous wave) power-detector circuitry in RF receivers.
These radio receivers are key products for Agilent customers manufacturing wireless communications equipment.
ET42803 SImplofoed Block Diagram
The CW power system is capable of characterizing a power detector against a traceable standard at power levels from -30 dBm to +40 dBm from 100 kHz to 4 GHz.
It does this with typical uncertainties of 0.1 dB (95.5% confidence).
This performance was achieved through careful attention to system design, mismatch contributions, harmonic content, and the accuracy and drift of the power
sensors and reference used. Algorithms for the calibration of the RF path losses within the test system and monitoring of performance parameters were also an
integral part of its operation.
To Automate or Not?
Back to top
As it turned out, the proposed automation project coincided with the need for improved RF detector specs for the flagship wireless communications test
product. An investigation commenced to assess the feasibility of the proposal. Initially, a measurement uncertainty (MU) analysis was carried out based on ISO
GUM(1) specs using the general s-parameter model for a 3-port device(2).
By comparing the weighted contributions of the various terms of this expression, potential problem areas were identified with the new system. The anticipated
problems included:
- An increase in length (1 m to 2 m) of the primary RF cable connecting the device under test (DUT) to the test system that would increase the uncertainty
arising from mismatch (VSWR) and repeatability (flexure), especially without careful selection.
- Push-fit N-type coaxial cable connections between the test system and the DUT, power sensors, and power references might show poorer repeatability than
torqued connectors.
- Standalone reference sources for 1 mW and 1 µW would be required since the manual calibration and zeroing of power sensors was no longer possible.
Solutions to these problems were identified in the form of:
- High-quality armored cabling.
- Specially adapted metrology-grade connectors mounted on a mobile "vehicle," with force control.
- The availability of new reference sources intended for standalone use.
The decision to automate was taken based on these approaches.
The Outcome
Back to top
While none of the anticipated problems caused any significant effects in the implementation of the automated test station, it was apparent that the performance
of the system was significantly less than that of those systems still operating in the manual environment. Moreover, the cause or causes of the degraded performance
weren't obvious.
The goal of a more robust and repeatable measurement hadn't been realized. The station had to be withdrawn from the Yellowstone environment.
But, to determine the causes of the poor system performance—without extensive experimentation involving a team of engineers—some form of reference
measurement was essential to compare the station's performance with known-good values.
The Introduction of SPC
Back to top
SPC had been used on production lines at Queensferry for some years, but in nearly every case the purpose had been to indicate trends or relative changes in the
performance of the test system over a period of time. A product representative of those being manufactured was generally used as the nominal standard for this testing.
However, this approach was insufficiently accurate or repeatable to identify the cause of the Power Test Station problems. A more rigorous testing scheme, one using
a more fundamental standard, was required.
It was also realized that SPC, which would help engineers to identify and then monitor/control test-system performance issues, had wider potential benefits.
In addition, a means of comparing a number of similar test systems against a single reference was highly desirable. A reference calibrated at a standards lab might
further provide direct traceability from production measurements to national standards.
A number of possible benefits were envisioned, including the following:
- Calibration of test-system RF paths would be driven by SPC rather than by a maintenance schedule
- Test system downtime would be reduced by an increased calibration interval
- It would maximize test-system yield (the target was greater than 99%)
- Clear identification and segregation of product and test-process problems would accrue
- Confidence in measurements would be maintained (an ISO17025 requirement)
There are three elements necessary to realize the foundation of SPC. Let's look at each one.
Measurement System Understanding
Back to top
A key aspect that SPC is required to verify is measurement uncertainty in the test system. This requires that you, as an engineer, have a comprehensive understanding
of the system operation down to the smallest contributor of uncertainty.
These contributors could include things such as switch and connector repeatability, or power-sensor drift. Your investment in time isn't insignificant, either!
The Reference or Gold Standard
Back to top
The proper selection of a so-called Gold Standard instrument is fundamental to SPC. The repeatability of the instrument must be significantly less than the test-system
uncertainty for the SPC process to add value. For this reason, the conventional approach of selecting a representative product is inappropriate in most cases.
There are a number of criteria that must be considered when selecting the Gold Standard.
- The instrument must operate over the parameters and range of the test system, and
- It must have a 2-sigma repeatability that's less than the system MU, and
- It must have calibration uncertainty that's appropriately less than the system MU, and
- To maintain confidence, its assigned calibration interval should be shorter than what's recommended by the manufacturer, and
- Cost must be considered (two units enable process continuity, for example, if one is being calibrated)
Gold Standard Operational Requirements
Back to top
There are also some Gold Standard operational requirements. The unit should be powered continually, even while not in use. Also, frequent maintenance should be performed
(i.e., connectors gauged and cleaned, fan filters cleaned, etc.).
Moreover, the Gold Standard must not be opened, adjusted, or used for diagnostics. It also must be handled, stored, and transported in a manner that will not affect the
calibration or physical condition of the instrument.
SPC Limits
Back to top
For warranted DUT specifications, the SPC limits should be no greater than the MU value. Type-B MU analysis will yield 95.5% confidence limits that are generally more
conservative, but apply to a number of test system of the same type throughout the recommended calibration interval.
Type-A analysis, on the other hand, may only be valid for a single system with defined system trace equipment, a defined Gold Standard, and a defined environment. Any
variation from system to system also needs to be accounted for.
Multiple Gold Standards will introduce further variation, and it may be necessary to add a term to the SPC limit values to account for this.
Test Times and Frequency
Back to top
Note that adding SPC to an existing test process will impact production time. The cost of running and maintaining an SPC methodology must be balanced against the risk to
quality through incorrectly calibrated end-product. It's therefore important to select the smallest number of test points that will fully exercise a system through its entire
operating range.
A similar argument applies to the frequency of Gold Standard runs. Consideration should be given to:
- Production-volume fluctuations
- Running daily for guaranteed DUT specifications
- Bi-daily or weekly intervals for noncritical specifications
SPC Failures
Back to top
An SPC failure can have many possible causes. These can include operator error, extraneous signals, contaminated connectors, cable wear, etc. The procedure for flagging a
failure must take into account all these mechanisms, and drive a corrective-action process.
In this real-world example, an SPC failure was defined as two consecutive runs that didn't pass all test points. In the automated environment, the Yellowstone controller
immediately put the test system off-line, preventing production throughput.
Reporting Results
Back to top
Trends in performance and SPC failures are more clearly seen by presenting results graphically. This helps you detect anomalies in system performance, the prediction of
system drift, or when a test system requires calibration.
Reviewing large data sets on a frequent basis is, however, time-consuming and prone to error, so test-support personnel are notified automatically by the system when an
SPC failure occurs. Such a reporting system balances the need for prompt remedial action with the collection of data for analysis.
In the next part of this article we'll discuss SPC implementation, and look at the calibration requirements of the Yellowstone station. We'll also offer some rules to
follow to successfully implement a metrology-based SPC system.
References
Back to top
- Guide to the Expression of Uncertainty in Measurement 1993 (E), International Organisation for Standardisation.
- R.A. Johnson, "Understanding Microwave Power Splitters," Microwave Journal (1975).
