Building an Automated Content Generation Pipeline with n8n, Llama 3, and Medium API Integration: From Idea Generation to Publication

Break free from manual content creation! By integrating n8n, Llama 3, and the Medium API, you can build an automated content generation pipeline from idea generation to final publication, saving time and effort while maximizing productivity. This article details the actual implementation methods and considerations.

1. The Challenge / Context

Content creators are constantly under pressure to discover new ideas and produce engaging content. The process from idea generation, drafting, editing, and publishing is time-consuming and involves many repetitive tasks. Especially for individual bloggers or small teams, this workload directly impacts the frequency and quality of content production. Furthermore, despite recent advancements in LLM technology, there is a lack of clear guidelines on how to efficiently integrate these technologies into content creation workflows. Therefore, automating repetitive tasks and streamlining the content creation process is crucial.

2. Deep Dive: n8n

n8n is a node-based workflow automation platform. Even users with limited coding experience can connect various services and build complex automation workflows through its intuitive interface. The core is "nodes." Each node performs a specific task, and data is transformed and processed as it passes through the nodes. n8n provides various nodes such as HTTP Request, Email, Database, CRM, and users can even develop custom nodes if needed. It offers flexibility for integrating external APIs like Llama 3 API and Medium API, and allows for the implementation of complex logic such as error handling, data transformation, and conditional branching. The power of n8n lies in its flexibility and extensibility. Users can freely configure and expand their workflows according to their needs.

3. Step-by-Step Guide / Implementation

Now, let's actually build an automated content generation pipeline integrating Llama 3 and the Medium API using n8n. Here is a step-by-step guide.

Step 1: n8n Installation and Setup

First, you need to install n8n. You can install it via Docker, npm, or cloud services. Using Docker is the simplest way.

docker run -d -p 5678:5678 -v ~/.n8n:/home/node/.n8n n8nio/n8n

Once installed, you can access the n8n UI by navigating to http://localhost:5678 in your web browser. Complete the initial setup (username, password).

Step 2: Prepare Llama 3 API Key

To use the Llama 3 API, you need an API key. Obtain a Llama 3 API key from the Meta AI website. It is recommended to manage the issued API key by setting it as an environment variable in n8n. You can set environment variables in n8n settings, for example, using `N8N_APP_SECRET` style.

Step 3: Prepare Medium API Integration Token

To use the Medium API, you need an Integration Token. Register a new application on the Medium Developer Portal and obtain an Integration Token. This token should also be set as an n8n environment variable.

Step 4: Create n8n Workflow: Idea Generation

Create a new n8n workflow and add the following nodes:

Cron node: Sets a periodic trigger. For example, you can set the workflow to run every day at 9 AM.
HTTP Request node (Llama 3): Calls the Llama 3 API to generate ideas.

The settings for the HTTP Request node are as follows:


        Method: POST
        URL: https://api.meta.com/llama3/generate
        Headers:
            Content-Type: application/json
            Authorization: Bearer {{ $env.LLAMA3_API_KEY }}
        Body:
            {
                "prompt": "Generate 5 blog post ideas about the latest technology trends. Include keywords such as 'artificial intelligence', 'machine learning', and 'automation'.",
                "max_tokens": 200
            }

This setting sends a prompt to the Llama 3 API: "Generate 5 blog post ideas about the latest technology trends. Include keywords such as 'artificial intelligence', 'machine learning', and 'automation'." and requests generation up to a maximum of 200 tokens. `{{ $env.LLAMA3_API_KEY }}` refers to the Llama 3 API key set as an environment variable.

Step 5: Create n8n Workflow: Content Generation and Refinement

Add the following nodes:

Function node: Extracts the list of ideas from the Llama 3 API response and separates each idea into individual items.
HTTP Request node (Llama 3): Generates a blog post draft using each idea.
Function node: Extracts the blog post draft from the Llama 3 API response.
HTTP Request node (Llama 3): Refines and edits the blog post draft. For example, it can correct grammatical errors and make sentences more natural.

Example settings for the content refinement HTTP Request node:


        Method: POST
        URL: https://api.meta.com/llama3/generate
        Headers:
            Content-Type: application/json
            Authorization: Bearer {{ $env.LLAMA3_API_KEY }}
        Body:
            {
                "prompt": "Refine and edit the following text: {{ $json.blogPost }}",
                "max_tokens": 500
            }

Step 6: Create n8n Workflow: Medium Publication

Add the following nodes:

HTTP Request node (Medium API): Calls the Medium API to publish the blog post.

Settings for the Medium API HTTP Request node:


        Method: POST
        URL: https://api.medium.com/v1/users/{{ $env.MEDIUM_USER_ID }}/posts
        Headers:
            Content-Type: application/json
            Authorization: Bearer {{ $env.MEDIUM_INTEGRATION_TOKEN }}
        Body:
            {
                "title": "{{ $json.title }}",
                "contentFormat": "html",
                "content": "{{ $json.refinedBlogPost }}",
                "publishStatus": "draft" // or "public"
                "tags": ["Artificial Intelligence", "Machine Learning", "Automation"]
            }

`{{ $env.MEDIUM_USER_ID }}` is the Medium user ID set as an environment variable, and `{{ $env.MEDIUM_INTEGRATION_TOKEN }}` is the Medium Integration Token set as an environment variable. Setting `publishStatus` to "draft" publishes it as a draft, while setting it to "public" publishes it immediately.

Important: When using the Medium API, the MEDIUM_USER_ID can be found in your profile page URL on the Medium website (e.g., the `@your_username` part in `https://medium.com/@your_username` becomes your user ID). If not set correctly, the API call will fail.

Step 7: Error Handling and Logging

It is important to add error handling logic to your workflow. You can use Try-Catch nodes to catch errors and configure them to send email notifications or log files when an error occurs. Additionally, using Log nodes to record the workflow execution process can help with debugging.

4. Real-world Use Case / Example

I use this workflow to automatically publish content to my personal blog more than twice a week. Previously, writing each post took at least 4 hours, but after building this workflow, I only need about 30 minutes per post for review and editing. I particularly saved time in the idea generation phase and was able to create more creative content based on the diverse ideas provided by Llama 3. Furthermore, I am conducting various experiments based on this workflow. For example, I plan to build workflows that automatically collect and summarize news articles on specific topics, or automatically generate social media posts.

5. Pros & Cons / Critical Analysis

Pros:
- Time Saving: Significantly reduces content creation time.
- Increased Productivity: Allows for faster creation of more content.
- Idea Generation Support: Provides diverse ideas through Llama 3.
- Automated Workflow: Automates repetitive tasks, increasing efficiency.
Cons:
- Initial Setup Complexity: Setting up n8n, Llama 3 API, and Medium API can be time-consuming.
- API Costs: Costs may incur depending on Llama 3 API usage.
- Content Quality Control: Automatically generated content must be reviewed and edited. Perfect automation is difficult.
- Technical Dependency: Dependent on external APIs, so workflows may need modification if APIs change.

6. FAQ

Q: Can I use other LLMs instead of Llama 3?
A: Yes, of course. You can also use OpenAI GPT models or other LLM APIs. You just need to change the URL and Body settings of the HTTP Request node to match the respective API.
Q: Can I use other blog platforms instead of Medium API?
A: Yes, if the platform provides an API, it's possible. For example, you could build a workflow using the WordPress API.
Q: What should I do if an error occurs during workflow execution?
A: Check the error logs in the n8n UI and add error handling logic using Try-Catch nodes. Also, double-check your API keys, user IDs, and environment variable settings.
Q: How should I manage Llama 3 API usage limits?
A: Check the Llama 3 API usage limits and consider adjusting the workflow execution frequency or reducing API usage. For example, you can run the workflow only once a day or reduce the length of the generated content.

7. Conclusion

Building an automated content generation pipeline by integrating n8n, Llama 3, and the Medium API is an effective way to save time and effort while maximizing productivity. Follow the steps presented in this guide to build your own workflow and automate your content creation process. Refer to the Llama 3 API documentation to experiment with various prompts and find the optimal workflow that suits your needs. Download n8n now and experience the world of automation!

Building an Automated Content Creation Pipeline Using n8n, Llama 3, and Medium API: From Idea Generation to Publication