.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/howto_logparser/plot_101_logparser_analyses.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_howto_logparser_plot_101_logparser_analyses.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_howto_logparser_plot_101_logparser_analyses.py:


Logparser: Predefined Analyses
==============================

In this section, we showcase various predefined analyses available in the log parser.
We utilize project files from the following benchmarks:
`ogs: Constant viscosity (Hydro-Thermal)
<https://www.opengeosys.org/docs/benchmarks/hydro-thermal/constant-viscosity>`_
and for the **staggered scheme** we use a prj from
`ogs tests: HT StaggeredCoupling HeatTransportInStationaryFlow
<https://gitlab.opengeosys.org/ogs/ogs/-/blob/master/Tests/Data/Parabolic/HT/StaggeredCoupling/HeatTransportInStationaryFlow/HeatTransportInStationaryFlow.prj>`_

.. GENERATED FROM PYTHON SOURCE LINES 14-16

.. code-block:: Python
   :dedent: 1










.. GENERATED FROM PYTHON SOURCE LINES 22-33

.. code-block:: Python

    import pandas as pd

    import ogstools as ot
    from ogstools.examples import (
        log_adaptive_timestepping,
        log_const_viscosity_thermal_convection,
        log_staggered,
    )

    pd.set_option("display.max_rows", 8)  # for visualization only








.. GENERATED FROM PYTHON SOURCE LINES 34-37

The preprocessing of logs remains consistent across all examples and
thoroughly explained in
:ref:`sphx_glr_auto_examples_howto_logparser_plot_102_logparser_advanced.py`.

.. GENERATED FROM PYTHON SOURCE LINES 37-42

.. code-block:: Python


    records = ot.logparser.parse_file(log_const_viscosity_thermal_convection)
    df_records = pd.DataFrame(records)
    df_log = ot.logparser.fill_ogs_context(df_records)








.. GENERATED FROM PYTHON SOURCE LINES 43-50

Analysis of iterations per time step
------------------------------------
For detailed explanation, refer to:
:ref:`sphx_glr_auto_examples_howto_logparser_plot_100_logparser_intro.py`.
(Section: Use predefined analyses)

:py:mod:`ogstools.logparser.analysis_time_step`

.. GENERATED FROM PYTHON SOURCE LINES 50-54

.. code-block:: Python

    df_ts_it = ot.logparser.time_step_vs_iterations(df_log)
    df_ts_it







.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>iteration_number</th>
        </tr>
        <tr>
          <th>time_step</th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>1</td>
        </tr>
        <tr>
          <th>1</th>
          <td>2</td>
        </tr>
        <tr>
          <th>2</th>
          <td>1</td>
        </tr>
        <tr>
          <th>3</th>
          <td>1</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
        </tr>
        <tr>
          <th>23</th>
          <td>9</td>
        </tr>
        <tr>
          <th>24</th>
          <td>17</td>
        </tr>
        <tr>
          <th>25</th>
          <td>23</td>
        </tr>
        <tr>
          <th>26</th>
          <td>20</td>
        </tr>
      </tbody>
    </table>
    <p>27 rows × 1 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 55-62

Analysis of computational efficiency by time step
-------------------------------------------------
The resulting table represents performance metrics for different parts of the simulation,
organized by time step. It utilizes :py:mod:`ogstools.logparser.analysis_time_step`.
displaying metrics such
as output time [s], step size [s], time step solution time [s], assembly time [s],
Dirichlet time [s], and linear solver time [s].

.. GENERATED FROM PYTHON SOURCE LINES 62-68

.. code-block:: Python


    df_ts = ot.logparser.analysis_time_step(df_log)
    df_ts = df_ts.loc[0]
    # Removing MPI_process (index=0) from result (all are 0) for serial log.
    df_ts






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>output_time</th>
          <th>step_size</th>
          <th>time_step_solution_time</th>
          <th>assembly_time</th>
          <th>dirichlet_time</th>
          <th>linear_solver_time</th>
        </tr>
        <tr>
          <th>time_step</th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0.004476</td>
          <td>NaN</td>
          <td>NaN</td>
          <td>0.000000</td>
          <td>0.000000</td>
          <td>0.000000</td>
        </tr>
        <tr>
          <th>1</th>
          <td>NaN</td>
          <td>1.000000e-07</td>
          <td>0.142436</td>
          <td>0.048265</td>
          <td>0.000944</td>
          <td>0.092496</td>
        </tr>
        <tr>
          <th>2</th>
          <td>NaN</td>
          <td>1.000000e-05</td>
          <td>0.041676</td>
          <td>0.013362</td>
          <td>0.000503</td>
          <td>0.027244</td>
        </tr>
        <tr>
          <th>3</th>
          <td>NaN</td>
          <td>1.000000e-03</td>
          <td>0.033367</td>
          <td>0.019580</td>
          <td>0.000451</td>
          <td>0.012804</td>
        </tr>
        <tr>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th>23</th>
          <td>NaN</td>
          <td>4.000000e+07</td>
          <td>0.531541</td>
          <td>0.177440</td>
          <td>0.004848</td>
          <td>0.343803</td>
        </tr>
        <tr>
          <th>24</th>
          <td>NaN</td>
          <td>2.000000e+08</td>
          <td>1.057740</td>
          <td>0.272943</td>
          <td>0.010794</td>
          <td>0.763421</td>
        </tr>
        <tr>
          <th>25</th>
          <td>NaN</td>
          <td>4.000000e+08</td>
          <td>3.149970</td>
          <td>0.666353</td>
          <td>0.014761</td>
          <td>2.459589</td>
        </tr>
        <tr>
          <th>26</th>
          <td>0.006782</td>
          <td>3.133499e+08</td>
          <td>2.406160</td>
          <td>0.550383</td>
          <td>0.012222</td>
          <td>1.833349</td>
        </tr>
      </tbody>
    </table>
    <p>27 rows × 6 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 69-70

Selecting specific metrics (3) and plotting using pandas plot function.

.. GENERATED FROM PYTHON SOURCE LINES 70-74

.. code-block:: Python

    df_ts[["assembly_time", "dirichlet_time", "linear_solver_time"]].plot(
        logy=True, grid=True
    )




.. image-sg:: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_001.png
   :alt: plot 101 logparser analyses
   :srcset: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    <Axes: xlabel='time_step'>



.. GENERATED FROM PYTHON SOURCE LINES 75-99

Analysis of convergence criteria - Newton iterations
----------------------------------------------------
The :py:mod:`ogstools.logparser.analysis_convergence_newton_iteration`
function allows for the examination of convergence criteria based on
Newton iterations. The resulting table provides convergence metrics for monolithic processes.
For details, refer to the documentation on
<`convergence_criterion
<https://doxygen.opengeosys.org/d4/d58/classnumlib_1_1convergencecriterion>`_ > defined in in the prj file.

* **|x|** is a norm of a vector of the global component (e.g. pressure, temperature, displacement).
* **|dx|** is the change of a norm of the global component between 2 iteration of non linear solver.
* **|dx|/|x|** is the relative change of a norm of the global component

For this example we had defined in the prj-file:

.. code-block:: python

   <convergence_criterion>
     <type>DeltaX</type>
     <norm_type>NORM2</norm_type>
     <abstol>1.e-3</abstol>
   </convergence_criterion>

The resulting table contains `|x|`, `|dx|` and `|dx|/|x|` at different time steps, processes and non linear solver iterations.

.. GENERATED FROM PYTHON SOURCE LINES 102-106

.. code-block:: Python


    ot.logparser.analysis_convergence_newton_iteration(df_log)







.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
          <th></th>
          <th>dx</th>
          <th>dx_x</th>
          <th>x</th>
        </tr>
        <tr>
          <th>time_step</th>
          <th>process</th>
          <th>iteration_number</th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th rowspan="2" valign="top">1</th>
          <th rowspan="2" valign="top">0</th>
          <th>1</th>
          <td>2.559100e+07</td>
          <td>2.530300e-02</td>
          <td>1.011400e+09</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0.000000e+00</td>
          <td>0.000000e+00</td>
          <td>1.011400e+09</td>
        </tr>
        <tr>
          <th>2</th>
          <th>0</th>
          <th>1</th>
          <td>0.000000e+00</td>
          <td>0.000000e+00</td>
          <td>1.011400e+09</td>
        </tr>
        <tr>
          <th>3</th>
          <th>0</th>
          <th>1</th>
          <td>0.000000e+00</td>
          <td>0.000000e+00</td>
          <td>1.011400e+09</td>
        </tr>
        <tr>
          <th>...</th>
          <th>...</th>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th rowspan="4" valign="top">26</th>
          <th rowspan="4" valign="top">0</th>
          <th>17</th>
          <td>9.385500e-03</td>
          <td>9.286300e-12</td>
          <td>1.010700e+09</td>
        </tr>
        <tr>
          <th>18</th>
          <td>1.646800e-03</td>
          <td>1.629400e-12</td>
          <td>1.010700e+09</td>
        </tr>
        <tr>
          <th>19</th>
          <td>2.994300e-03</td>
          <td>2.962700e-12</td>
          <td>1.010700e+09</td>
        </tr>
        <tr>
          <th>20</th>
          <td>5.881000e-04</td>
          <td>5.818800e-13</td>
          <td>1.010700e+09</td>
        </tr>
      </tbody>
    </table>
    <p>136 rows × 3 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 107-115

Staggered
~~~~~~~~~
The resulting table provides convergence criteria for staggered coupled processes,
utilizing :py:mod:`ogstools.logparser.analysis_convergence_coupling_iteration`
Logs are generated from running
`ogs benchmark: HeatTransportInStationaryFlow
<https://gitlab.opengeosys.org/ogs/ogs/-/blob/master/Tests/Data/Parabolic/HT/HeatTransportInStationaryFlow/HeatTransportInStationaryFlow.prj>`_


.. GENERATED FROM PYTHON SOURCE LINES 115-122

.. code-block:: Python

    records = ot.logparser.parse_file(log_staggered)
    df_records = pd.DataFrame(records)
    df_log = ot.logparser.fill_ogs_context(df_records)

    # Only for staggered coupled processes !
    ot.logparser.analysis_convergence_coupling_iteration(df_log)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th></th>
          <th></th>
          <th>dx</th>
          <th>dx_x</th>
          <th>x</th>
        </tr>
        <tr>
          <th>time_step</th>
          <th>coupling_iteration</th>
          <th>coupling_iteration_process</th>
          <th></th>
          <th></th>
          <th></th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th rowspan="4" valign="top">1</th>
          <th rowspan="2" valign="top">1</th>
          <th>0</th>
          <td>0.35337</td>
          <td>0.17246</td>
          <td>2.049</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0.00000</td>
          <td>0.00000</td>
          <td>1271200.000</td>
        </tr>
        <tr>
          <th rowspan="2" valign="top">2</th>
          <th>0</th>
          <td>0.00000</td>
          <td>0.00000</td>
          <td>2.049</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0.00000</td>
          <td>0.00000</td>
          <td>1271200.000</td>
        </tr>
        <tr>
          <th>...</th>
          <th>...</th>
          <th>...</th>
          <td>...</td>
          <td>...</td>
          <td>...</td>
        </tr>
        <tr>
          <th rowspan="2" valign="top">49</th>
          <th rowspan="2" valign="top">1</th>
          <th>0</th>
          <td>0.00000</td>
          <td>0.00000</td>
          <td>8.293</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0.00000</td>
          <td>0.00000</td>
          <td>1271200.000</td>
        </tr>
        <tr>
          <th rowspan="2" valign="top">50</th>
          <th rowspan="2" valign="top">1</th>
          <th>0</th>
          <td>0.00000</td>
          <td>0.00000</td>
          <td>8.379</td>
        </tr>
        <tr>
          <th>1</th>
          <td>0.00000</td>
          <td>0.00000</td>
          <td>1271200.000</td>
        </tr>
      </tbody>
    </table>
    <p>102 rows × 3 columns</p>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 123-130

Analysis of model time and clock time
-------------------------------------
The :py:mod:`ogstools.logparser.model_and_clock_time` function allows to
examine needed iterations, clock time, and step size over model time per
attempted time step. This is especially useful to analyze the runtime
behaviour of a simulation which employs adaptive time stepping. The following
example failed as the simulation reached the minimal allowed time step size.

.. GENERATED FROM PYTHON SOURCE LINES 132-138

.. code-block:: Python

    records = ot.logparser.parse_file(log_adaptive_timestepping)
    df_records = pd.DataFrame(records)
    df_log = ot.logparser.fill_ogs_context(df_records)
    df_t = ot.logparser.model_and_clock_time(df_log)
    df_t[["step_size", "clock_time", "iterations"]].plot(grid=True, subplots=True)




.. image-sg:: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_002.png
   :alt: plot 101 logparser analyses
   :srcset: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    array([<Axes: xlabel='model_time'>, <Axes: xlabel='model_time'>,
           <Axes: xlabel='model_time'>], dtype=object)



.. GENERATED FROM PYTHON SOURCE LINES 139-141

To get an overview of the convergence behavior of the nonlinear solver over
the entire simulation we can plot the relative error as a heatmap.

.. GENERATED FROM PYTHON SOURCE LINES 143-145

.. code-block:: Python

    fig = ot.logparser.plot_convergence(df_log, "dx_x")




.. image-sg:: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_003.png
   :alt: plot 101 logparser analyses
   :srcset: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 146-148

We can also plot this data over the model time. Note, that the last timesteps
are so small, that they are not visible anymore here:

.. GENERATED FROM PYTHON SOURCE LINES 150-152

.. code-block:: Python

    fig = ot.logparser.plot_convergence(df_log, "dx_x", x_metric="model_time")




.. image-sg:: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_004.png
   :alt: plot 101 logparser analyses
   :srcset: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 153-159

Further we can calculate the convergence order and plot it in the same manner.
In order to estimate the convergence order we need to take into account
multiple values and thus cannot assign each iteration a convergence order.
Only for iterations `i` of `i  >= n` an order is calculated and plotted.
See: :py:func:`~ogstools.logparser.common_ogs_analyses.convergence_order_per_ts_iteration`
for more info.

.. GENERATED FROM PYTHON SOURCE LINES 161-162

.. code-block:: Python

    fig = ot.logparser.plot_convergence_order(df_log)



.. image-sg:: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_005.png
   :alt: plot 101 logparser analyses
   :srcset: /auto_examples/howto_logparser/images/sphx_glr_plot_101_logparser_analyses_005.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 2.111 seconds)


.. _sphx_glr_download_auto_examples_howto_logparser_plot_101_logparser_analyses.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_101_logparser_analyses.ipynb <plot_101_logparser_analyses.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_101_logparser_analyses.py <plot_101_logparser_analyses.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_101_logparser_analyses.zip <plot_101_logparser_analyses.zip>`