The translate tools core is a kit of translation primitives.
The package contains translators, scheduler for translation tasks, tex-to-speech (TTS), some utils and standardized types for a build a large systems.
Use this package if you need default implementations of translation primitives. Use a types of this package, to implement your own entities and to make it compatible with a numerous utils implemented by other people.
Feel free to open a new issue and request a primitives that needs in your project and will useful for other people.
Install package npm install @translate-tools/core
This package provides CJS and ESM modules both. This docs contains CJS examples, that can be used for both NodeJS and Browser, but if you need to use ESM modules (to make tree-shaking are effective for example), you can add /esm prefix for any paths, after package name:
// Example with import a CommonJS module
import { GoogleTranslator } from '@translate-tools/core/translators/GoogleTranslator';// Example with import a ECMAScript module
import { GoogleTranslator } from '@translate-tools/core/esm/translators/GoogleTranslator';If you update package to a new major version, check migration guides to do it smooth.
Directory translators contains a translators interfaces and default implementations.
Translator purpose is translate text from one language to another.
Translators in this package uses 2-letter ISO 639-1 language codes, but in your translator implementation you can use ISO 639-2 language codes too.
Translators have 2 public methods, translate for translate single text and translateBatch for translate a multiple texts per one request. Method translateBatch may have better translation quality for some use cases, because some translator implementations may not just translate every single string independently, but see a context of all strings.
Example with google translator
import { GoogleTranslator } from '@translate-tools/core/translators/GoogleTranslator';
const translator = new GoogleTranslator();
// Translate single string
translator
.translate('Hello world', 'en', 'de')
.then((translate) => console.log('Translate result', translate));
// Translate multiple strings
translator
.translateBatch(
['Hello world', 'Some another text to translate', 'Yet another text'],
'en',
'de',
)
.then((translatedTexts) => console.log('Translate result', translatedTexts));For use with nodejs you have to specify user agent. In most cases for nodejs, translator will work incorrectly with not set User-Agent header.
import { GoogleTranslator } from '@translate-tools/core/translators/GoogleTranslator';
const translator = new GoogleTranslator({
headers: {
'User-Agent':
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36',
},
});Sometimes you may want to use custom fetcher, to send requests through proxy server, implement retries or cache, etc.
For example, some translators API is not available in browser, due to CSP policy, for this cases you may use some CORS proxy to fix the problem. With this option, translator will use proxy to send requests, keep in mind that those who own the proxy server you use, can see your requests content.
You may pass your implementation of Fetcher function to option fetcher.
import { GoogleTranslator } from '@translate-tools/core/translators/GoogleTranslator';
import { Fetcher } from '@translate-tools/core/utils/Fetcher';
import { basicFetcher } from '@translate-tools/core/utils/Fetcher/basicFetcher';
// Extend `basicFetcher` with use CORS proxy for all requests
const fetcher: Fetcher = async (url, options) => {
// Use some CORS proxy service as prefix
return basicFetcher('https://corsproxy.io/?' + url, options);
};
const translator = new GoogleTranslator({ fetcher });You may use any fetcher under the hood, like Axios, Got, or even postMessage.
Package includes translators implementations for most popular services.
Uses a free API version of service translate.google.com
Exports 2 implementations GoogleTranslator and GoogleTranslatorTokenFree, that have the same features, but uses different API endpoints, so you can choose what implementation better for you.
import {
GoogleTranslator,
GoogleTranslatorTokenFree,
} from '@translate-tools/core/translators/GoogleTranslator';
const translator = new GoogleTranslator({
headers: {
'User-Agent':
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36',
},
});
translator
.translate('Hello world', 'en', 'de')
.then((translate) => console.log('Translate result', translate));
const translator2 = new GoogleTranslatorTokenFree({
headers: {
'User-Agent':
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36',
},
});
translator2
.translate('Hello world', 'en', 'de')
.then((translate) => console.log('Translate result', translate));Uses a free API of service translate.yandex.ru
Uses a free API https://github.com/TartuNLP/translation-api built with TartuNLP's public NMT engines API docs: https://api.tartunlp.ai/translation/docs Demo: https://translate.ut.ee/
Uses a free API of service https://www.reverso.net/text-translation
Unstable: this translator are not stable and placed in unstable subdirectory. Keep in mind the not stable translators may be removed any time.
Uses API of https://www.deepl.com/translator
This translator requires to provide API key
import { DeepLTranslator } from '@translate-tools/core/translators/DeepLTranslator';
const translator = new DeepLTranslator({
apiKey: '820c5d18-365b-289c-e63b6fc7e1cb:fx',
});
translator
.translate('Hello world', 'en', 'de')
.then((translate) => console.log('Translate result', translate));Uses API of https://github.com/LibreTranslate/LibreTranslate
This translator requires to provide API endpoint and API key for some not free API instances. See an instances list to find a public instances.
Unstable: this translator are not stable and placed in unstable subdirectory. Keep in mind the not stable translators may be removed any time.
Options
apiHostoptional - url to API endpoint. Default:https://translate.terraprint.co/translateapiKeyoptional - API key
import { LibreTranslateTranslator } from '@translate-tools/core/translators/unstable/LibreTranslateTranslator';
const freeTranslator = new LibreTranslateTranslator({
apiHost: 'https://translate.argosopentech.com/translate',
});
const localDeployedTranslator = new LibreTranslateTranslator({
apiHost: 'http://localhost:9077/translate',
});
const paidTranslator = new LibreTranslateTranslator({
apiHost: 'https://libretranslate.com/translate',
apiKey: '76c1d41c-0a9a-5667-e9d169746e1e',
});Fake translator for tests, that returns original string with added prefix.
Module translators/Translator contains translator interfaces.
Interface describes instance members for a translator
interface TranslatorInstanceMembers {
/**
* Translate text
* @returns Translated string
*/
translate(text: string, langFrom: langCodeWithAuto, langTo: langCode): Promise<string>;
/**
* Translate texts array
* @returns Array with translated strings and same length as input array
* @returns Texts that did not translated will replaced to `null`
*/
translateBatch(
text: string[],
langFrom: langCodeWithAuto,
langTo: langCode,
): Promise<Array<string | null>>;
/**
* Check string or array of stings to exceeding a limit
*
* It need for modules with complexity logic of encoding a strings.
* For example, when in `translateBatch` text merge to string
* and split to chunks by tokens with ID: `<id1>Text 1</id1><id2>Text 2</id2>`
*
* Here checked result of encoding a input data
* @returns number of extra chars
*/
checkLimitExceeding(text: string | string[]): number;
/**
* Check supporting of translate direction
*/
checkDirection?: (langFrom: langCodeWithAuto, langTo: langCode) => boolean;
/**
* Max length of string for `translate` or total length of strings from array for `translateBatch`
*/
getLengthLimit(): number;
/**
* Delay between requests that required by translator to a correct work
*/
getRequestsTimeout: () => number;
}Interface describes static members, useful to choose most suitable translator in a list by features
interface TranslatorStaticMembers {
/**
* Public translator name to displaying
*/
readonly translatorName: string;
/**
* Is required API key for this module
*/
isRequiredKey(): boolean;
/**
* Is it supported value `auto` in `langFrom` argument of `translate` and `translateBatch` methods
*/
isSupportedAutoFrom(): boolean;
/**
* Array of supported languages as ISO 639-1 codes
*/
getSupportedLanguages(): langCode[];
}Interface describes translator constructor, that have members of TranslatorStaticMembers and construct an object with type TranslatorInstanceMembers
Interface describes default options for any translator constructor.
Some translators may ignore some options.
type TranslatorOptions<O extends Record<any, any> = {}> = O & {
/**
* Access key for requests to translator API
*/
apiKey?: string;
/**
* Union text array to 1 request (or more, but less than usualy anyway).
*
* Option for reduce the number of requests, but it can make artefacts in translated text.
*
* Some modules may not support this feature.
*/
useMultiplexing?: boolean;
/**
* Additional headers for requests
*/
headers?: Record<string, string>;
/**
* Custom fetcher
*/
fetcher?: Fetcher;
};Directory scheduling contains primitives to schedule translation, that allow to batch few translation requests to one API request.
Scheduler are extremely useful for cases when your code intensive calls translation methods, to avoid API rate limit errors and to collect few requests for short time to one request.
Scheduler have standard interface, to allow you implement your own scheduler for you use case
export interface ISchedulerTranslateOptions {
/**
* Identifier to grouping requests
*/
context?: string;
/**
* Priority index for translate queue
*/
priority?: number;
/**
* Use direct translate for this request if it possible
*/
directTranslate?: boolean;
}
export interface IScheduler {
translate(
text: string,
from: langCodeWithAuto,
to: langCode,
options?: ISchedulerTranslateOptions,
): Promise<string>;
}Scheduler API looks like translator with one public method translate. User calls this method any times with any frequency, and scheduler executes requests by its own plan.
Basic scheduler placed at scheduling/Scheduler;
import { GoogleTranslator } from '@translate-tools/core/translators/GoogleTranslator';
import { Scheduler } from '@translate-tools/core/scheduling/Scheduler';
// We use google translator API for translate requests
const translator = new GoogleTranslator();
const scheduler = new Scheduler(translator, { translatePoolDelay: 100 });
// This text will not translated immediately, instead it will be added to suit batch in queue,
// and will translated after 100ms after last update this batch
scheduler.translate('Some text for translate', 'en', 'de', { priority: 1 });
// The same with this request, but this text will translated first, because have most high priority
scheduler.translate('Text that must be translated ASAP', 'en', 'de', { priority: 10 });
// Texts with same priority will be batched
scheduler.translate('Some text for translate', 'en', 'de', { priority: 1 });interface Scheduler {
new (translator: TranslatorInstanceMembers, config?: SchedulerConfig): IScheduler;
}
interface SchedulerConfig {
/**
* Number of attempts for retry request
*/
translateRetryAttemptLimit?: number;
/**
* If true - rejected requests will use direct translate
*/
isAllowDirectTranslateBadChunks?: boolean;
/**
* Length of string for direct translate.
*
* null for disable the condition
*/
directTranslateLength?: number | null;
/**
* Delay for translate a chunk. The bigger the more requests will collect
*/
translatePoolDelay?: number;
/**
* When chunk collect this size, it's will be instant add to a translate queue
*
* null for disable the condition
*/
chunkSizeForInstantTranslate?: number | null;
/**
* Pause between handle task batches
*
* It may be useful to await accumulating a task batches in queue to consider priority better and don't translate first task batch immediately
*
* WARNING: this option must be used only for consider priority better! Set small value always (10-50ms)
*
* When this option is disabled (by default) and you call translate method for texts with priority 1 and then immediately for text with priority 2, first request will have less delay for translate and will translate first, even with lower priority, because worker will translate first task immediately after delay defined by option `translatePoolDelay`
*/
taskBatchHandleDelay?: null | number;
}Scheduler with cache placed at scheduling/SchedulerWithCache and useful for cases when you needs to cache your translations.
This class is just a decorator over Scheduler that allow you to use standard object ICache with scheduler.
import { GoogleTranslator } from '@translate-tools/core/translators/GoogleTranslator';
import { Scheduler } from '@translate-tools/core/scheduling/Scheduler';
import { SchedulerWithCache } from '@translate-tools/core/scheduling/SchedulerWithCache';
import { ICache } from '@translate-tools/core/utils/Cache';
// Dummy implementation of cache
class CacheInRam implements ICache {
private cache: Record<string, string> = {};
private buildKey(text: string, from: string, to: string) {
return [text, from, to].join('-');
}
async get(text: string, from: string, to: string) {
const cacheKey = this.buildKey(text, from, to);
return this.cache[cacheKey] ?? null;
}
async set(text: string, translate: string, from: string, to: string) {
const cacheKey = this.buildKey(text, from, to);
this.cache[cacheKey] = translate;
}
async clear() {
this.cache = {};
}
}
// We use google translator API for translate requests
const translator = new GoogleTranslator();
const scheduler = new Scheduler(translator, { translatePoolDelay: 100 });
const cache = new CacheInRam();
const schedulerWithCache = new SchedulerWithCache(scheduler, cache);
schedulerWithCache.translate('Some text for translate', 'en', 'de');
schedulerWithCache.translate('Some another text', 'en', 'de');interface SchedulerWithCache {
new (scheduler: Scheduler, cache: ICache): IScheduler;
}
/**
* Interface of cache
*/
export interface ICache {
/**
* Get translation
*/
get: (text: string, from: string, to: string) => Promise<string | null>;
/**
* Set translation
*/
set: (text: string, translate: string, from: string, to: string) => Promise<void>;
/**
* Clear cache
*/
clear: () => Promise<void>;
}Directory tts contains text to speech modules, to make audio from a text.
TTS modules returns buffer with audio.
import { GoogleTTS } from '@translate-tools/core/tts/GoogleTTS';
const tts = new GoogleTTS();
tts.getAudioBuffer('Some demo text to speak', 'en').then(({ buffer, type }) => {
// Play audio from fetched `ArrayBuffer`
const audio = new Audio();
audio.src = URL.createObjectURL(new Blob([buffer], { type }));
audio.play();
});Uses a free API version of service translate.google.com
Uses API of https://github.com/thedaviddelta/lingva-translate
This module supports option apiHost to provide API endpoint, see an instances list to find a public instances. Default API host: https://translate.plausibility.cloud.
/**
* Object with audio data
*/
export type TTSAudioBuffer = {
/**
* Audio mimetype
*/
type: string;
/**
* Buffer contains audio bytes
*/
buffer: ArrayBuffer;
};
/**
* TTS instance members
*/
export interface TTSProviderProps {
/**
* Get blob with audio
* @param text text to speak
* @param language text language
* @param options optional map with preferences to generate audio
*/
getAudioBuffer(
text: string,
language: string,
options?: Record<string, string>,
): Promise<TTSAudioBuffer>;
}
/**
* TTS constructor members
*/
export interface TTSProviderStaticProps {
getSupportedLanguages(): string[];
}
/**
* Text to speech module
*/
export type TTSProvider = TTSProviderStaticProps & {
new (...args: any[]): TTSProviderProps;
};Module languages contains utils to work with language codes.
Predicate to check is language code are valid ISO-639v1 code or not
Predicate to check is language code are valid ISO-639v2 code or not
Returns language codes for selected version of ISO-639
You can support the project to help it maintain independence and high quality:
- tell others about this package
- suggest new ideas, features and elegant ways to make package better
- help us by trying to reproduce unconfirmed bug reports
- help us address bugs. Just ping us in issues and start work on PRs
Also, you can donate to us to vote with money for goal prioritization, to add some feature or fix some bug as soon as possible. Just donate with any method below, then leave transaction details in issue comments, or send email to contact@vitonsky.net with issue number or feature request description. For significant donations, we will start work on your request as soon as possible.
- Monero (XMR): 861w7WuFGecR7SMpuf7GX9BBUgGJb1Xdx8z5pCpMrKY2ZeZAzS3mwZeQeJGV5RPpu35fr5dURSm587ewpHYGzNuGKGroQnD
- Bitcoin (BTC): bc1q2krassq0sa2aphkx37zn374lfjnthr5frm6s7y
- Ethereum (ETH), Tether USDT (ERC-20): 0x2463d84F46c131886CaE457412e8B6eaBc0b91a7
- Tron (TRC), Tether USDT (TRC-20): TQezzyzkfMCPJRdnYxNXrUfPj3s7kDeMBL