Tag Archives: model

Plot covariance estimates in a GaussianMixture cluster

covariance

Moving covariance matrix functions from GMM to GaussianMixture in sklearn

So in sklearn 0.18 the GMM model is deprecated and looks to be removed in 0.20. The replacement is GaussianMixture available from sklearn.mixture. I have some code that currently uses GMM and needed to port it to GaussianMixture. One of the major convienence features to GMM are accessors to mean, covars, etc. The Covariance matrix itslef is critical for many of the clustering applications that would motivate the original use of GMM. This post shows how to port that covariance accessor to the promoted GaussianMixture implementation.

Covariance Matrix

Remember that the covariance matrix is a matrix representing the covariances between all of the elements between two vectors, X and Y:

Cov[X,Y]=E[(X − E[X])(Y − E[Y])]=E[XY] − E[X]E[Y]

An alternative form, perhaps more expressive when considering the matrix forms that we are dealing with here and using Σ as the covariance matrix, and μ is the mean of any random vector X:

Σ = E[(X − μ)(X − μ)T ] = E[XXT ] − μμT

Python Intelligent Algorithms

Working with a GMM clustering over the iris dataset, there is a dependency on the make_ellipses method published by Ron Weiss ronweiss@gmail.com.
The most non-trivial move required to shift from GMM to GaussianMixture involves:

v, w = np.linalg.eigh(gmm._get_covars()[n][row_idx[:, None], col_idx])

which needs to shift to

v, w = np.linalg.eigh(gmm.covariances_[n][row_idx[:, None], col_idx])

in order to work with the newer model package.
The full listing of the function as I have ported it is then:

def make_ellipses(gmm, ax, x, y):
"""
Extracts a covariance matrix in 2D from a higher dimensional feature space.
Calculates an ellipse along maximal variance in a GMM object in 2D,
both direction and the respective magnitude, i.e., the eigenvector and eigenvalue
of the covariance matrix. It writes the resulting ellipse onto an existing pyplot plot.
:param gmm: sklearn GaussianMixture object
:param ax: plot axis - the 2D subset of the full feature space
:param x: the first dimension of the 2D plot axis
:param y: the second dimension of the 2D plot axis
:return:
"""
for n, color in enumerate('rgb'):
    row_idx = np.array([x, y])
    col_idx = np.array([x, y])
    # FIXME GMM has method _get_covars not present in GaussianMixture
    #v, w = np.linalg.eigh(gmm._get_covars()[n][row_idx[:, None], col_idx])
    v, w = np.linalg.eigh(gmm.covariances_[n][row_idx[:, None], col_idx])
    u = w[0] / np.linalg.norm(w[0])
    angle = np.arctan2(u[1], u[0])
    angle = 180 * angle / np.pi  # convert rads to degrees
    v *= 9
    ell = mpl.patches.Ellipse(gmm.means_[n, [x, y]], v[0], v[1], 180 + angle, color=color)
    ell.set_clip_box(ax.bbox)
    ell.set_alpha(0.5)
    ax.add_artist(ell)

For an excellent discussion on the use of the ellipses for plotting the covariances of your GaussianMixture see GMM covariances

Some analysis of Royce's own concerns with the Waterfall model

Over at the *Art of SW Dev* is a very good post giving some historical analysis on [Waterfall vs Agile](http://sinnema313.wordpress.com/2010/01/16/waterfall-vs-agile/), using Royce’s original paper, and a good understanding of what is agile in the present day. It finds that Royce has been very unfairly mischaracterized. He found many flaws in the Waterfall (some suggest labelling this as *single-pass waterfall to give Royce credit for wanting iteration), and wanted things like full test coverage, people over process, etc.

It’s worth a parse and some thought.

More from the Master Switch

I [wrote](http://seanmehan.globat.com/blog/2011/06/10/the-minimal-energy-well-of-accepted-thought/) the other day about some thoughts about local energy minima as a function of innovation, sparked by Timothy Wu’s *The Master Switch*, one of my current reads. I wanted to capture a fabulous quote and story about one of the iterations of Wu’s *cycle*, dealing with the *radio telephone*. Continue reading More from the Master Switch