# BLEU Score

## BLEU Score

### What is the BLEU Score?

The BLEU (Bilingual Evaluation Understudy) Score is an evaluation metric for machine translation that measures the quality of translated text by comparing it to human-translated reference texts. The BLEU Score is based on the modified n-gram precision, which counts the number of n-gram matches between the machine-translated output and the reference translations, taking into account both the closeness of the translation and the length of the translated text.

### How is the BLEU Score calculated?

The BLEU Score is calculated using the following steps:

1. Compute the modified n-gram precision for each n-gram order (usually up to 4-grams) by counting the number of n-grams in the machine-translated output that also appear in the reference translations, and normalize by the total number of n-grams in the output.
2. Calculate the geometric mean of the modified n-gram precisions.
3. Apply a brevity penalty to penalize shorter translations: If the output translation is shorter than the reference translations, the BLEU Score is multiplied by a factor that decreases as the output length decreases.
4. The final BLEU Score ranges from 0 to 1, with higher scores indicating better translations.

### Example of calculating BLEU Score in Python using NLTK:

``````from nltk.translate.bleu_score import sentence_bleu

# Example of reference translations and machine translation
reference_translations = [["the cat is on the mat", "there is a cat on the mat"]]
machine_translation = "the cat is on the mat"

# Calculate the BLEU Score
bleu_score = sentence_bleu(reference_translations, machine_translation.split())
print("BLEU Score:", bleu_score)
``````

## Byte Pair Encoding (BPE)

### What is Byte Pair Encoding?

Byte Pair Encoding (BPE) is a data compression algorithm that iteratively replaces the most frequent pair of consecutive bytes with a single, unused byte. In natural language processing (NLP), BPE has been adapted as a subword tokenization method to handle out-of-vocabulary words effectively, by splitting them into smaller subword units. BPE-based tokenization has been used in various NLP models, such as OpenAI’s GPT-2 and Google’s BERT.

### How does Byte Pair Encoding work in NLP?

In NLP, BPE tokenization is performed as follows:

1. Initialize the vocabulary with unique characters in the dataset.
2. Count the frequency of all character pairs in the dataset.
3. Merge the most frequent pair, creating a new token.
4. Repeat steps 2 and 3 for a predefined number of iterations or until no more merges are possible.

### Example of Byte Pair Encoding in Python:

Here’s a simple implementation of BPE tokenization using Python:

``````from collections import Counter, defaultdict

def get_stats(vocab):
pairs = Counter()
for word, freq in vocab.items():
symbols = word.split()
for i in range(len(symbols) - 1):
pairs[symbols[i], symbols[i + 1]] += freq
return pairs

def merge_vocab(pair, vocab_in):
vocab_out = {}
bigram = ' '.join(pair)
replacement = ''.join(pair)
for word, freq in vocab_in.items():
new_word = word.replace(bigram, replacement)
vocab_out[new_word] = freq
return vocab_out

def bpe_tokenization(vocab, num_merges):
for i in range(num_merges):
pairs = get_stats(vocab)
if not pairs:
break
best_pair = pairs.most_common(1)
vocab = merge_vocab(best_pair, vocab)
return vocab

# Example usage
raw_vocab = {'l o w </w>': 5, 'l o w e r </w>': 2, 'n e w e s t </w>': 6, 'w i d e s t </w>': 3}
num_merges = 10
bpe_vocab = bpe_tokenization(raw_vocab, num_merges)
print(bpe_vocab)
``````

This example demonstrates a simple BPE tokenization process on a given vocabulary, performing 10 merge operations.