.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/plot_fisher.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_plot_fisher.py>`
        to download the full example code or to run this example in your browser via JupyterLite.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_plot_fisher.py:


=========================
Fisher's criterion in LDA
=========================

.. currentmodule:: fastcan

In this examples, we will demonstrate the canonical correlation coefficient
between the features ``X`` and the one-hot encoded target ``y`` has equivalent
relationship with Fisher's criterion in LDA (Linear Discriminant Analysis).

.. GENERATED FROM PYTHON SOURCE LINES 12-16

.. code-block:: Python


    # Authors: The fastcan developers
    # SPDX-License-Identifier: MIT








.. GENERATED FROM PYTHON SOURCE LINES 17-22

Prepare data
------------
We use ``iris`` dataset and transform this multiclass data to multilabel data by
one-hot encoding. Here, drop="first" is necessary; otherwise, the transformed target
is not full column rank.

.. GENERATED FROM PYTHON SOURCE LINES 22-33

.. code-block:: Python


    from sklearn import datasets
    from sklearn.preprocessing import OneHotEncoder

    X, y = datasets.load_iris(return_X_y=True)
    # drop="first" is necessary; otherwise, the transformed target is not full column rank
    y_enc = OneHotEncoder(
        drop="first",
        sparse_output=False,
    ).fit_transform(y.reshape(-1, 1))








.. GENERATED FROM PYTHON SOURCE LINES 34-39

Compute Fisher's criterion
--------------------------
The intermediate product of ``LinearDiscriminantAnalysis`` in ``sklearn`` is
Fisher's criterion, when ``solver="eigen"``. However, it does not provide an interface
to export it, so we reproduce it manually.

.. GENERATED FROM PYTHON SOURCE LINES 39-58

.. code-block:: Python


    import numpy as np
    from scipy import linalg
    from sklearn.covariance import empirical_covariance
    from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

    clf = LinearDiscriminantAnalysis(solver="eigen").fit(X, y)
    Sw = clf.covariance_  # within scatter
    St = empirical_covariance(X)  # total scatter
    Sb = St - Sw  # between scatter
    fishers_criterion, _ = linalg.eigh(Sb, Sw)

    fishers_criterion = np.sort(fishers_criterion)[::-1]
    n_nonzero = min(X.shape[1], clf.classes_.shape[0] - 1)
    # remove the eigenvalues which are close to zero
    fishers_criterion = fishers_criterion[:n_nonzero]
    # get canonical correlation coefficients from convert Fisher's criteria
    r2 = fishers_criterion / (1 + fishers_criterion)








.. GENERATED FROM PYTHON SOURCE LINES 59-64

Compute SSC
-----------
Compute the sum of squared canonical correlation coefficients (SSC). It can be found
that the result obtained by :class:`FastCan`/CCA (Canonical Correlation Analysis) is
the same as LDA.

.. GENERATED FROM PYTHON SOURCE LINES 64-71

.. code-block:: Python


    from fastcan import FastCan

    ssc = FastCan(4, verbose=0).fit(X, y_enc).scores_.sum()

    print(f"SSC from LDA: {r2.sum():5f}")
    print(f"SSC from CCA: {ssc:5f}")




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    SSC from LDA: 1.191899
    SSC from CCA: 1.191899





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.066 seconds)


.. _sphx_glr_download_auto_examples_plot_fisher.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: lite-badge

      .. image:: images/jupyterlite_badge_logo.svg
        :target: ../lite/lab/index.html?path=auto_examples/plot_fisher.ipynb
        :alt: Launch JupyterLite
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_fisher.ipynb <plot_fisher.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_fisher.py <plot_fisher.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_fisher.zip <plot_fisher.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_