By
Safran Federal Systems
It’s Hard to Keep Time and Data Straight
Multiple receivers testing in parallel Time tagging and error computation in real-time
Data in numerous formats-apples-to-apples comparison
Comprehensive data logging with simple quick look reports
Agile testing - test and adapt quickly using results
Final reports and conclusions automatically generated Leveraging PANACEA, it’s possible to script entire tests, control hardware, and log data.
Getting Results with Less Effort - Making Field Tests Fun and Productive
GPS vulnerability testing began with the creation of the NAVWAR program and investigates navigation system performance in the presence of interference signals. This testing is crucial for the design, development, fielding, sustainment and mission planning of the DoD as well as commercial PNT systems. The testing is typically conducted in three steps involving modeling and simulation, lab/ chamber testing, and live fire field test exercises.
Safran professionals have been engaged in NAVWAR initiatives since the early 2000s, supporting efforts across all three stages of development. This involvement has extended to working alongside U.S. defense organizations tasked with advancing navigation warfare capabilities. While these government entities lead such efforts, numerous product groups and vendors are required to test and validate their systems to ensure reliable performance for their customers. Safran has partnered with these stakeholders by providing proven expertise, test methods, and tools for system evaluation. With compounding test variables from evolving threats to complex integrations, the testing process and ability to capture meaningful results is daunting.
PANACEA was built to support hardware-in-the-loop testing regardless of whether it’s in the lab, chamber or at a live fire field test. Beginning with lab testing, PANACEA reduces the risks of field tests and identifies the projected results allowing for organizations to concentrate on the tests that matter most. These pre-run scenarios can then be prepared for live fire field testing. Similar hardware can then be used to reduce field testing costs while also streamlining the test execution and errors that may result from human control. Using PANACEA,timelines can be simple or complex with the ability to change signal interferences quickly and accurately. The units under test are also controlled, configured, and reporting data to the PANACEA computer, allowing for the time synchronization between live fire events and the logged receiver data. The true power is the knowledge gained in real-time, allowing testers to quickly quantify test success or failures with the ability to be agile and retest scenarios on the fly.
Lab Testing
Prior testing in the lab is paramount to a successful field exercise. Understanding the systems capabilities and pre-running the scenarios of interest provide a baseline to form hypotheses from. Thousands of scenarios can automatically be tested and then modified to arrive at the scenarios of highest interest. Engineers can then use these tested and calibrated scenarios at field events for increased success. These baselines also support a benchmark for field testers to understand if things are “going well” in the field. Lab testing also vets the data collection process and enables final report baselines which often forces the testers to rethink the questions that need to be asked, and the data required to answer those questions.
Configuring Units Under Test (UUTs)
The most important part of any test is the UUT. Ensuring the UUT is configured correct such that data can be gathered on its performance while not impacting the test. This setup could be identical to the lab setup, again reducing risk and cost. One major difference when field testing versus lab testing is that the timeline continues to move forward instead of a fixed start and stop time. PANACEA allows the user to select the start time to be time synchronized with a receiver/live sky orto use the PC system time. This time scale allows all the receiver data to be gathered and coordinated to permit apples-to-apples comparison. A truth reference is also included (A GNSS receiver tracking a non-effected constellation) to permit real-time error computations while dynamic or at arbitrary points. The other option would be to presurvey the points and use those as the reference in your scenario.
Taking the Lab to the Field
n preparation for field testing, the inevitable question of what hardware will be used comes up. In some cases, lab equipment must stay in a sterile environment, but in many cases, lab equipment can be outfitted to support field tests as well. The value is more than the added cost of having to purchase two separate systems. Using common equipment reduces risk in commonality between tests as well as operations, maintenance and tester training in operating different equipment. PANACEA has been built to support the direct injection of RF as well as Over-The-Air (OTA) transmission ensuring that field tests match what was run in the lab.
Data Reduction and Dissemination
While the focus of a field test is generally on the test articles, participants and timelines, a larger consideration should be the data collection process and how that data will be used to arrive at conclusions. Many hours are spent on building test log formats, time scales, and data entry forms. These are still beneficial to provide cross checks, but the focus should be on automating the data collection and the ability to quickly and confidently analyze the data. Time stamping is crucial, and in some cases, external references must be used. These files should also have a consistent format to enable easy comparisons and analysis without question. These files along with the analysis artifacts need to be made available to support the report and permit future testers to dig into the data in preparation for future tests. PANACEA and Panorama provide a cohesive data collection and reporting capability that enables testers to show the data in real-time providing near instant after action reporting. By providing senior leaders and test supervisors results within hours after the test completion, decisions regarding how to proceed with the following tests can be made as opposed to blindly testing and collecting data. The quick look reports can then be created the same week and provide results for a reduced timeline. In most cases, the longer the reporting takes and the further from the test event the analyst is pushed, the more likely the report will be flawed.
Summary
The ability to conduct them as efficiently as possible has always been a function of understanding the systems in the planning process and the ability to quickly and cohesively gather data during the event. PANACEA was designed specifically for these actions and has become a primary resource for many organizations conducting NAVWAR exercises. Using common hardware, processes, and reporting in all phases of testing, PANACEA provides the lowest cost, risk and schedule while enabling the highest results. Maybe the least considered attribute to testing in this manner is the tester feedback. Making the tests “fun” and less stressful on test engineers also promotes better results. Stress free test engineers tend to focus on the results and conclusions and less on the laborious button pushing, data filing, and recollection of what happened during the tests.