/* required for Doxygen */

/** @file */ 

/** \mainpage APEX: Autonomic Performance Environment for eXascale

\section copyright Copyright

\verbinclude LICENSE.txt

\section overview Overview

One of the key components of the XPRESS project is a new approach to
performance observation, measurement, analysis and runtime decision making in
order to optimize performance. The particular challenges of accurately
measuring the performance characteristics of ParalleX applications requires a
new approach to parallel performance observation. The standard model of
multiple operating system processes and threads observing themselves in a
first-person manner while writing out performance profiles or traces for
offline analysis will not adequately capture the full execution context, nor
provide opportunities for runtime adaptation within OpenX. The approach taken
in the XPRESS project is a new performance measurement system, called
(Autonomic Performance Environment for eXascale). APEX will include methods for
information sharing between the layers of the software stack, from the hardware
through operating and runtime systems, all the way to domain specific or legacy
applications. The performance measurement components will incorporate relevant
information across stack layers, with merging of third-person performance
observation of node-level and global resources, remote processes, and both
operating and runtime system threads.

\section introduction Introduction
\subsection interfaces Interfaces

Essentially, APEX is both a measurement system for introspection, as well
as a Policy Engine for modifying runtime behavior based on the 
observations. While APEX has capabilities for generating profile data
for post-mortem analysis, the key purpose of the measurement is to 
provide support for policy enforcement. To that end, APEX is designed
to have very low overhead and minimize perturbation of runtime worker thread
productivity. APEX supports both start/stop timers and either 
event-based or periodic counter samples. Measurements are taken synchronously,
but profiling statistics and internal state management is performed by
(preferably lower-priority) threads distinct from the running application. The
heart of APEX is an event handler that dispatches events to registered
listeners within APEX.  Policy enforcement can trigger synchronously when
events are triggered by the OS/RS or application, or can occur asynchronously
on a periodic basis.

APEX is a library written in C++, and has both C and C++ external interfaces.
While the C interface can be used for either language, some C++ applications
prefer to work with namespaces (i.e.  apex::*) rather than prefixes (i.e.
apex_*). All functionality is supported through both interfaces, and the C
interface contains inlined implementations of the C++ code.

While the designed purpose for APEX is supporting the current and future needs 
of ParalleX runtimes within the XPRESS project (such as HPX3, HPX5),
experimental support is also available for introspection of current runtimes
such as OpenMP. APEX could potentially be integrated into other runtime
systems, such as any of a number of lightweight task based systems. The
introspection provided by APEX is intended to be in the third-person model,
rather than traditional first-person, per-thread/per-process application
profile or tracing measurement.  APEX is designed to combine information from
the OS, Runtime, hardware and application in order to guide policy decisions.

For distributed communication, APEX provides an API to be implemented for 
the required communication for a given application. An MPI implementation
is provided as a reference, and both HPX3 and HPX5 implementations have
been implemented. In this way, APEX is integrated into the observed
runtime, and asyncrhonous communication is provided at a lower priority,
in order to minimize perturbation of the application.

The direct links to each API are here:
<ul>
<li>C API : @ref apex.h </li>
<li>C++ API : @ref apex </li>
</dl>

\section usermanual User Manual

For a complete user manual, please see <a href="http://khuck.github.io/xpress-apex">the APEX documentation.</a>


\section acknowledgements Acknowledgements

Support for this work was provided through Scientific Discovery through
Advanced Computing (SciDAC) program funded by U.S. Department of Energy, Office
of Science, Advanced Scientific Computing Research (and Basic Energy
Sciences/Biological and Environmental Research/High Energy Physics/Fusion
Energy Sciences/Nuclear Physics) under award numbers DE-SC0008638,
DE-SC0008704, DE-FG02-11ER26050 and DE-SC0006925.

**/
