lightopic

The lightopic package helps make BERTopic model deployments smaller and lighter. When installed without any of the optional extras, lightopic provides methods for using a trained and serialised model to transform new documents (i.e. label them with a topic).

You can also install with the bertopic package to train your model.

Submodules

Classes

Lightopic

The Lightopic class is to be used with a trained topic model

Package Contents

class lightopic.Lightopic

The Lightopic class is to be used with a trained topic model that has been serialised appropriately (see docs for lightopic.LightBERTopic.save_lightopic). Methods will transform new data by reducing the embeddings or putting them into topics.

load(model_directory: str) → None
reduce_embeddings(embeddings: numpy.ndarray, output_type: str = 'reduced') → numpy.ndarray

Reduce the dimensions of an input array of embeddings.

Parameters:

  • embeddings (numpy.ndarray) – An array of embeddings.

  • output_type (str, optional) – Set the dimensions of the output. Defaults to “reduced”, which will reduce to the dimensions used for clustering. (If you used the BERTopic defaults this will be 5.) Setting this to “2d” will output 2-dimensional vectors that you can use for plotting.

Raises:

  • ValueError – if output_type is not either “reduced” or “2d”.

  • ValueError – the dimension of the embeddings must be compatible with the Lightopic model, i.e. it must match either the full dimension of the original embeddings (self.len_full_embedding) or that of the reduced embeddings (self.len_umap_embedding).

Returns:

The reduced embeddings.

Return type:

numpy.ndarray

transform(embeddings: numpy.ndarray, calculate_probabilities: bool = True) → tuple[numpy.ndarray, numpy.ndarray]

Transform new documents with the model.

Parameters:

  • embeddings (numpy.ndarray) – Array of embeddings

  • calculate_probabilities (bool, optional) – Return probabilities for all topics? Defaults to True.

Raises:

ValueError – the dimension of the embeddings must be compatible with the Lightopic model, i.e. it must match either the full dimension of the original embeddings (self.len_full_embedding) or that of the reduced embeddings (self.len_umap_embedding).

Returns:

The topic assignments

and probabilities.

Return type:

tuple[numpy.ndarray, numpy.ndarray]