Automated Information Extraction and Analyses Scheme for Investigating the Non-linear Cascading Effects in Highly Dependent Networks

PI: Anita Raja

RA:  Mohammad Hasan

Collaborator:  Professor Maureen Brown, UNCC.

Sponsor: NPS

Research Goal:

This research intends to proactively study the network characteristics and to investigate the cascading effects in highly dependent MDAP (Major Defense Acquisition Programs) clusters. It will look into the dynamics of the joint space of MDAPs and how it affects the acquisition process. The aim is to use the Network Theory to understand the key properties of the MDAP funding network and to use Markov Decision Process (MDP) to design an appropriate decision-theoretic model for decision making and planning for cascading effects in the face of uncertainty.

Background:

To cope with the evolving nature of threats, DoD (Department of Defense) went through transformation toward joint capabilities to enable government agencies to cope with new challenges and threats, and to adapt accordingly. The aim of these joint capabilities was to share resources among services, agencies and nations. This transformation required successful material solutions that could establish joint demands. In short, successful joint acquisition was considered to be a pre-condition to achieve the goal of the transformation.

We identify that the acquisition arena has been evolved from being program-centric towards network-centric, and plan to understand the nature of interdependencies among the nodes in the network and their potential consequences. We will examine the interdependencies in the context of a successful and a not-so-successful MDAP.  Based on Defense Acquisition executive Summary (DAES) reports, Selected Acquisition Reports (SAR) reports and other Program Element (PE) documents, we will construct funding interdependency networks in which MDAPs share common PE accounts. As illustration, Figure 1 shows data interdependencies of the 78 MDAPs in 2005. Our research question evolves around formulation of appropriate metrics that provide early indication of the acquisition risks of interdependent programs. Research findings should be able to improve early assessment of program development resources, establish more realistic program thresholds and highlight areas of risk that may have escaped management attention.

Figure 1: MDAP Data Interdependencies in 2005

Our Approach:

We observe dependencies among different MDAPs who share common funding/data partners. Figure 2 shows the evolution of interdependency and complexity of the MDAP funding network.

 

       MDAP relationships in 1997                                        MDAP relationships in 2007

        Figure 2: Growing interdependency and growing complexity in MDAP network

 
We plan to define funding networks for successful and not-so-successful MDAPs and want to understand the issues that cause a program to falter, and also to understand the decision mechanisms behind successful programs. We will study the DAES reports for the program of interest and would focus on the quality of the forecasting mitigating of the problems. In case of the failure of the forecasting we would investigate the local issues pertaining to the boundary of the program itself responsible for the slip in forecasting. For the absence of local issues we would search the non-local issues that reside outside the program boundary and may have originated and propagated from a neighboring program.

 

In short, our research objectives are three-fold:

1.      Examine and compare the network characteristics of interdependent regions belonging to successful and not so successful MDAP programs to augment our current work in “what-if” analyses.

2.      Automate the data extraction and analysis process by leveraging algorithms for decision support as well as image and text analysis.

3.       Design a Markovian decision-theoretic model & continue to identify the challenges in acquiring the data from the government and program managers


Data Analysis:

In our recent work [Raja et. al. 2012], we investigated a small funding network MDAP_A consisting of five MDAPs (figure 3).

 
Figure 3: Funding Network of MDAP_A in 2010

 

MDAP_A, a communications program initiated in 2004 whose program name has been scrubbed for confidentiality purposes, is the central MDAP for our study. This program is our focus because (a) the data available about this program is significant; (b) between the years 2006 to 2010, it experienced multiple APB (Acquisition Program Baseline) breaches and increase in %PAUC (Per Unit acquisition Cost), making it a critical node for reference. Using information about the funding partners of MDAP_A, we define a logical funding network shown in figure 3.The other nodes in the graph are neighbor programs of MDAP_A that share common funding agencies. The link between any two nodes refers to the funding relationship and serves as interface among the programs. These links illustrate the interdependent regions of the case study network. The funding network allows us to do a detailed study of the performance of the member nodes and to understand the cascading effects in.

Result:

Our findings indicate that MDAP-related data characteristics support the multiple perspective study of perturbations and it is possible to recast the study of cascading effects as a sequential decision problem. We also note that it is crucial to consider the uncertainty in action outcomes in the decision-making process and that a non-local perspective may help explain a performance breach in situations where a solely local perspective does not. These observations provide evidence supporting our conjecture that MDPs are a good avenue to study interdependencies in the MDAP network and to capture early indicators of interdependency risk. Finally, we have captured the informational value in the existing data and challenges inherent in the data collection process with respect to their role in isolating risks and initiating appropriate government oversight methods.

Understanding the Characteristics of Data:

We tried to understand the characteristics of the existing data and identified the inherent challenges in the data-collection process. The available offers significant insight about each individual program as well as their interdependency relationships. DAES reports, which are published monthly, provide a granular view of the local issues pertaining to the program and the mitigation actions that has been taken to resolve the issues. SAR files, on the other hand, provide a quantitative depiction of the program status on the basis of accrued breaches, increase in %PAUC, cost and funding figures. This resource helps us for comparative quantitative analyses and to gain insight about the cascading effects. We, however, observed that the structure of the data is not uniform. None of the reports of the joint programs capture the joint space or non-local region directly. Some programs do not report (DAES) regularly and some of them even stopped reporting. Therefore, the structure and availability of data are the critical issues that need to be addressed to design a complete and accurate decision-theoretic model for the funding network.

Observations:

We studied the available DAES and SAR files of two programs (MDAP_A and MDAP_B) of the network from 2006 to 2010, in an effort to identify cascading effect in the funding network. We tried to understand the local as well as non-local issues that led the programs towards breach condition. Following are the summary of observations from this process:

O1: Design of MDAPs relies on cutting edge technology. It appears that the contractor either underestimates or cannot accurately estimate the technical challenges and the amount of funding required to accomplish the tasks.

O2: Programs are observed to be suffered greatly by budget cut. Sometimes it does not receive required amount of funding from Government (Congressional committee), which delays the schedule, and as a consequence cost increases.

O3: Lack in procurement funding is another cause that leads to cost and funding problems.

O4: Analyses of the local issues and the fact that some of the issues are recurrent indicate that either the root cause of the problem is not captured in the DAES documents or that the cause is exogenous of the program boundary.

O5. Analyses of SAR files, on the other hand, offer some insight about the interdependency of the programs.

O6. The observed instance of possible cascading effect in the funding network motivates us to design an automated scheme (Markovian decision-theoretic model) where this problem could be recasted a sequential decision making problem.

References:

Brown, M. M., Flowe, R. and Raja, A. (2010) Acquisition Risks in a World of Joint Capabilities. Proceedings of Naval Postgraduate Schools 7th Annual Acquisition Research Symposium, May 12-13, Monterey, CA.

Raja, A., and Lesser, V. (2007). A framework for meta-level control in multi-agent systems. Autonomous Agents and Multi-Agent Systems 15(2):147–196.

Raja, A. and Lesser, V. (2008). Coordinating Agents' Meta-level Control. Proceedings of AAAI 2008  Workshop on Metareasoning: Thinking about Thinking, pp 106-112, Chicago, IL. July 2008.

Raja, A., Hasan, M. R. & Brown, M. M. (2012). Facilitating Decision Choices with Cascading Consequences in Interdependent Networks. Proceedings of Naval Postgraduate Schools 9th Annual Acquisition Research Symposium, May 15-17, Monterey, CA.

URL: http://www.researchsymposium.com/?page=overview&conference=25

URL: http://www.researchsymposium.com/?page=overview&c=25&conference=31