Falcon LLM: The New King of Open Source LLMs

Image by editor

We’re seeing big language models (LLMs) spitting out more and more chatbots for us every week. However, it is difficult to know which is the best, progress in each, and which is the most useful.

HuggingFace has an Open LLM Leaderboard that tracks, rates and ranks LLMs as they are released. They use a unique framework for testing generative language models on a variety of evaluation tasks.

Recently, LLaMA (Large Language Model Meta AI) has been at the top of the leaderboard and they just dropped a new pre-built LLM – Falcon 40B.

Image by HuggingFace Open LLM Leaderboard

The Falcon LLM was founded and built by the Technology Innovation Institute (TII), a company that is part of the Abu Dhabi Government’s Advanced Technology Research Council. The government oversees technological research throughout the UAE, with a team of scientists, researchers and engineers focused on delivering transformative technologies and discoveries in science.

Falcon-40B is a fundamental LLM with a 40B setting that trains on a trillion tokens. The Falcon 40B is an autoregressive decoder only model. An autoregressive decoder model only means that the model was trained to predict the next token from a sequence of previous tokens. A good example of this is the GPT model.

The Falcon architecture has been shown to significantly outperform GPT-3 at only 75% of the training computing budget, as well as only requiring ? Compute during inference.

Data quality at scale was an important focus of the Institute for Technology Innovation team, as we know that LLMs are very sensitive to the quality of training data. The team built a data pipeline that scaled to tens of thousands of CPU cores for fast processing and was able to extract high-quality content from the Internet using extensive filtering and deduplication.

They also have another smaller version: the Falcon-7B, which has a 7B setting, trained on a 1500B token. Falcon-40B-Instruct and Falcon-7B-Instruct models are also available if you are looking for a ready-made chat model.

What can the Falcon 40B do?

Like other LLMs, the Falcon 40B can:

• Create creative content
• Solving difficult problems
• Customer service operations
• virtual assistants
• Language translation
• sentiment analysis.
• Reduce and automate “repetitive” tasks.
• Help Emirati companies become more efficient

How was the Falcon 40B trained?

To train 1 trillion tokens required 384 GPUs on AWS for two months. We trained on 1000 billion tokens of RefinedWeb, a massive English web database built by TII.

The pre-training data consisted of collecting public data from the web using CommonCrawl. The team went through a thorough filtering phase to remove machine-generated text, and aggregated adult content as well as any duplicates to create a pre-training database of nearly five trillion tokens.

Built on top of CommonCrawl, the RefinedWeb database has shown models to achieve better performance than models trained on curated datasets. RefinedWeb is also multimodal.

Once it was ready, Falcon was validated against open source benchmarks such as EAI Harness, HELM and BigBench.

They have open-sourced the Falcon LLM to the public, making the Falcon 40B and 7B more accessible to researchers and developers, as it is based on Apache License version 2.0.

The LLM, once only for research and commercial use, has now become open source to meet the global demand for inclusive access to AI. It is now royalty-free for commercial use restrictions as the UAE is ready to change the challenges and boundaries within artificial intelligence and how it will play an important role in the future.

To build an ecosystem for collaboration, innovation and knowledge sharing in the world of AI, Apache 2.0 provides security and secure open source software.

If you want to try a simpler version of the Falcon-40B that’s better suited for general chatbot-style instructions, you’ll want to use the Falcon-7B.

So let’s get started…

If you haven’t already, install the following packages:

!pip install transformers
!pip install einops
!pip install accelerate
!pip install xformers

Once you’ve installed these packages, you can proceed to run the code provided for the Falcon 7-B Instruct:

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model = "tiiuae/falcon-7b-instruct"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
    device_map="auto",
)
sequences = pipeline(
   "Girafatron is obsessed with giraffes, the most glorious animal on the face of this Earth. Giraftron believes all other animals are irrelevant when compared to the glorious majesty of the giraffe.\nDaniel: Hello, Girafatron!\nGirafatron:",
    max_length=200,
    do_sample=True,
    top_k=10,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
)
for seq in sequences:
    print(f"Result: seq['generated_text']")

As the best open source model available, the Falcon took LLaMA’s crown and people are amazed by its highly optimized architecture, unique open source license and available in two sizes: 40B and 7B configurations.

have you tried If you have, let us know what you think in the comments.

There is a niche is a data scientist, freelance technical writer and community manager at KDnuggets. He is particularly interested in data science career advice or tutorials and theory-based knowledge about data science. He also wants to explore different ways in which artificial intelligence is/can benefit from human longevity. An eager learner looking to expand his technical knowledge and writing skills while helping others.