Article Recommender with Text Embedding, Classification, and Extraction

Embeddings can capture the meaning of a piece of text beyond keyword-matching. In this article, we will build a simple news article recommender system that computes the embeddings of all available articles and recommend the most relevant articles based on embeddings similarity.

We will also make the recommendation tighter by using text classification to recommend only articles within the same category. We will then extract a list of tags from each recommended article, which can further help readers discover new articles.

All this will be done via three Cohere API endpoints stacked together: Embed, Classify, and Generate.

We will implement the following steps:

1. Find the most similar articles to the one currently reading using embeddings.
2. Keep only articles of the same category using text classification.
3. Extract tags from these articles.
4. Show the top 5 recommended articles.

1. Find the most similar articles to the one currently reading using embeddings

Throughout this article, we'll use the BBC news article dataset as an example [Source]. This dataset consists of articles from a few categories: business, politics, tech, entertainment, and sport. Here are some example articles:

1.2 Turn articles into embeddings

The first thing we need to do is to turn each article's text into embeddings. An embedding is a list of numbers that our models use to represent a piece of text, capturing its context and meaning. We do this by calling Cohere’s Embed endpoint, which takes in texts as input and returns embeddings as output.

articles = df_inputs['Text'].tolist()

output = co.embed(
            model ='large',
            texts = articles)
embeds = output.embeddings

1.3 Pick one article and find the most similar articles

Next, we pick any one article to be the one the reader is currently reading (let's call this the target) and find other articles with the most similar embeddings (let's call these candidates) using cosine similarity.

Cosine similarity is a metric that measures how similar two sequences of numbers are (embeddings in our case), and we compute it for each target-candidate pair.

from sklearn.metrics.pairwise import cosine_similarity
 
def get_similarity(target,candidates):
  # Calculate cosine similarity
  similarity_scores = cosine_similarity(target,candidates)
 
  # Sort by descending order in similarity
  similarity_scores = list(enumerate(similarity_scores))
  similarity_scores = sorted(similarity_scores, key=lambda x:x[1], reverse=True)
 
  # Return similarity scores
  return similarity_scores

Using Article ID 70 as an example target article, here’s what we get:

Target:

Candidates:

2. Keep only articles of the same category using text classification

In the example above (Article ID 70 as the target), we see that the top 5 most similar articles given by the system are very relevant. The target is a football/soccer article, and the system duly recommended very similar articles despite this dataset also containing articles from other sports like tennis and rugby.

However, not all of them are. The fourth recommended article is not a sports article, but rather politics. Reading the text, it's likely because the target is an article about a clash of individuals (i.e. anger about a racism fine), which also happens to be what the politics article is about (i.e. disagreement over an apology). So these two articles' meanings are similar in this way, captured in the embeddings.

Perhaps we can enhance the system by only recommending articles of the same category. For this, let's build a news category classifier.

2.1 Build a classifier

We use Cohere’s Classify endpoint to build a news category classifier, classifying articles into five classes: Business, Politics, Tech, Entertainment, and Sport.

A typical text classification model requires hundreds/thousands of data points to train, but with this endpoint, we can build a classifier with as few as five examples per class.

To build the classifier, we need a set of examples consisting of text (news text) and labels (news category). The BBC News dataset happens to have both (columns 'Text' and 'Category'), so this time we’ll use the categories for building our examples.

The Classify endpoint needs a minimum of 5 examples for each category, which we will sample randomly from the dataset. We have 5 categories, so we will have a total of 25 examples.

# Get classifications via the Classify endpoint
def classify_text(text,examples):
  classifications = co.classify(
    model='medium',
    taskDescription='',
    outputIndicator='',
    inputs=[text],
    examples=examples
    )
  return classifications.classifications[0].prediction

2.2 Measure its performance

Before actually using the classifier, let's first test its performance. Here we take another 100 data points as the test dataset and the classifier will predict the classes i.e. news category.

# Predicted classes
predictions = df_test['Text'].apply(classify_text, args=(examples,)).tolist()
 
# Actual classes
actual = df_test['Category'].tolist()
 
# Compute metrics on the test dataset
accuracy = accuracy_score(actual, predictions)

We get a good accuracy score of 91% (more details in the notebook), so the classifier is ready to be implemented in our recommender system.

3. Extract tags from these articles

We now proceed to the tags extraction step. Compared to the previous two steps, this step is not about sorting or filtering articles, but rather enriching them with more information.

We do this by prompting Cohere’s Generate endpoint with a few examples of text and its tags. We then feed the articles from the classifier step and the endpoint will generate the corresponding tags.

There is more than one way to construct the prompt, depending on what you'd like to extract. In my case, the tags I'd like to extract are primarily the names of a person, company, or organization, and perhaps also some generic keywords. That was the idea behind the example tags I put in the prompt, which you can see on the Cohere Playground screenshot below:

We call the endpoint by specifying a few settings, and it will generate the corresponding extractions.

# Get extractions via the Generate endpoint
def extract_tags(complete_prompt):
  prediction = co.generate(
    model='xlarge',
    prompt=complete_prompt,
    max_tokens=30,
    temperature=0.3,
    k=0,
    p=1,
    frequency_penalty=0,
    presence_penalty=0,
    stop_sequences=["--"],
    return_likelihoods='NONE')
  return prediction.generations[0].text

4. Show the Top 5 recommended articles

Let's now put everything together for our article recommender system.

First, we select the target article and compute the similarity scores against the candidate articles. Next, we filter the articles via classification. Finally, we extract the keywords from each article and show the recommendations.

Keeping to Article ID 70 as an example target article, here’s what we get:

You are reading...

You might also like...

Here we see how the classification and extraction steps have improved our recommendation outcome.

First, now the politics article doesn't get recommended anymore. Second, now we have the tags related to each article being generated.

Let’s try a couple of other articles in business and tech and see the output.

You are reading...

You might also like...

You are reading...

You might also like...

In conclusion, this demonstrates an example of how we can stack multiple NLP endpoints together to get an output much closer to our desired outcome.

In practice, hosting and maintaining multiple models can turn quickly into a complex activity. But by leveraging Cohere endpoints, this task is reduced to a simple API call.