Local LLMs: Using ollama, LLaMA3, and Apple’s Silicon Processor to summarize News and Gauge Sentiment

4 min readMay 12, 2024

I recently stumbled upon ollama, a very interesting project, still in early stages, that allow users to effortlessly run popular open-source Large Language Models (LLMs) directly on their devices, in my case a Mac Mini M1.

What’s more, ollama offers an Application Programming Interface (API) for generating completions, conversational responses, and embeddings — making it a valuable tool for developers and researchers alike.

The idea

Motivated by this concept, I embarked on a personal project to harness the power of language models for news analysis. Specifically, I created a small-scale application that extracts articles from reputable news sources, summarizes each piece in concise form, categorizes their sentiment as positive, negative, or neutral, and generates an index of overall positivity across the entire dataset.

The result you can visit and see below.

News summarizer link

The tools

ollama & LLaMA3 & M1

Getting started with ollama is remarkably straightforward. Simply download the installer and place it in your Applications folder. After installation, you can launch the application like any other native app on your device. Upon startup, ollama will automatically open port 11434 for API access, allowing you to seamlessly integrate it into your workflow. I decided to use LLaMA3, to download just run:

ollama pull llama3

And to run LLaMA3, execute the following:

ollama run llama3

In addition to the terminal prompt, ollama also gives you an API, to test it, run the following:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3",
  "stream": false,
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ]
}'

and you will get a response like the one below:

{
  "model": "registry.ollama.ai/library/llama3:latest",
  "created_at": "2024-05-05T14:13:43.416799Z",
  "message": {
    "role": "assistant",
    "content": "Hello! How are you today?"
  },
  "done": true,
  "total_duration": 5191566416,
  "load_duration": 2154458,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 383809000,
  "eval_count": 298,
  "eval_duration": 4799921000
}

scrape-it

scrape-it is a remarkably simple yet incredibly powerful web scraping tool developed in JavaScript. When working with one particular source, I utilized the following code snippet to extract all the homepage articles:

  async getHomepageArticles(): Promise<article[] | null> {
    const { data } = (await scrapeIt(this.newsUrl, {
      // Fetch articles
      articles: {
        listItem: ".c-article",
        data: {
          // article title
          title: {
            attr: "data-name",
          },
          // article date
          date: {
            attr: "data-publicacion",
            convert: (x) => new Date(x),
          },
          // article id
          id: {
            attr: "data-id",
          },
          // article category
          category: {
            attr: "data-category",
          },
          // article url
          url: {
            selector: "h3 > a",
            attr: "href",
          },
        },
      },
    })) as any;
    return data.articles;
  }

Putting it all together

The final solution is very simple and has 3 components:

The news summarizer (node)
The frontend (React & AWS Amplify)
The database (AWS S3)

The news summarizer

My Mac Mini M1 executes this node app at two-hour intervals. It downloads news articles from the last couple of hours, condenses the information into concise summaries, analyzes their sentiment and calculates a positivity index.

The frontend

A React-based web page, deployed through AWS Amplify, retrieves and visualizes the analysis files in an intuitive and user-friendly format.

The database

All analysis files are stored in Amazon S3 in JSON format, allowing for historical record-keeping. My objective is to leverage this dataset in the future to facilitate querying and trend discovery using Amazon Athena, ultimately enabling data-driven insights.

Final thoughts

Projects like ollama, coupled with advancements in low-cost, low-voltage GPU technology found in Apple Silicon and other processors, are revolutionizing the development process for local language models (LLMs). This democratization of AI capabilities allows developers to create and run inexpensive LLMs without relying on cloud-based solutions or token purchases. As a result, we can expect an unprecedented explosion of innovative AI applications over the next few years.