feat: add Spaces support (creation, Janus integration, STT/TTS pipeline)

[slkzgm]

Pull Request: Add Twitter Spaces Support

Overview

This PR adds comprehensive Twitter Spaces support to the twitter-scraper package, enabling programmatic creation and interaction with Twitter Spaces, including bi-directional audio streaming and plugin capabilities.

Dependencies

Added these dependencies:

{
  "@roamhq/wrtc": "*",
  "ws": "*",
}

Core Features

• Full Twitter Spaces lifecycle management
• WebRTC integration via Janus Gateway
• Bi-directional audio streaming (48kHz, mono)
• Plugin system with included plugins:
  • STT/TTS with GPT interaction
  • Audio recording
  • Real-time monitoring

Technical Architecture

Core Components

src/
├── spaces/
│   ├── Space.ts             # Main Space management
│   ├── JanusClient.ts       # WebRTC/Janus integration
│   ├── plugins/             
│   │   ├── SttTtsPlugin.ts  # Speech-to-text + GPT + TTS
│   │   ├── RecorderPlugin.ts# Audio recording
│   │   └── MonitorPlugin.ts # Real-time monitoring
│   ├── test.ts             # Example usage & testing
│   └── types/              # Type definitions

Plugin System

MonitorAudioPlugin

Real-time audio monitoring using ffplay.

const monitorPlugin = new MonitorAudioPlugin(48000); // sampleRate
space.use(monitorPlugin);

Features:

• Real-time PCM playback via ffplay
• Supports custom sample rates
• Automatic cleanup on space end

RecordToDiskPlugin

Records Space audio to disk.

const recordPlugin = new RecordToDiskPlugin();
space.use(recordPlugin);

Features:

• Raw PCM recording
• Default output: /tmp/speaker_audio.raw
• Automatic file stream management

SttTtsPlugin

Speech-to-text + GPT + Text-to-speech pipeline.

const sttTtsPlugin = new SttTtsPlugin();
space.use(sttTtsPlugin, {
  openAiApiKey: process.env.OPENAI_API_KEY,
  elevenLabsApiKey: process.env.ELEVENLABS_API_KEY,
  sttLanguage: 'en',
  gptModel: 'gpt-3.5-turbo',
  silenceThreshold: 50
});

Features:

• OpenAI Whisper STT
• ChatGPT interaction
• ElevenLabs TTS
• Silence detection
• Automatic audio streaming

Usage Example

async function main() {
  // 1. Initialize scraper
  const scraper = new Scraper();
  await scraper.login(
    process.env.TWITTER_USERNAME!,
    process.env.TWITTER_PASSWORD!
  );

  // 2. Create Space
  const space = new Space(scraper);
  
  // 3. Add plugins
  space.use(new RecordToDiskPlugin());
  space.use(new SttTtsPlugin(), {
    openAiApiKey: process.env.OPENAI_API_KEY,
    elevenLabsApiKey: process.env.ELEVENLABS_API_KEY
  });

  // 4. Configure Space
  const config: SpaceConfig = {
    mode: 'INTERACTIVE',
    title: 'Test Space',
    description: 'Testing spaces integration',
    languages: ['en']
  };

  // 5. Initialize and start
  const broadcastInfo = await space.initialize(config);
  console.log('Space URL:', broadcastInfo.share_url.replace('broadcasts', 'spaces'));

  // 6. Handle events
  space.on('occupancyUpdate', (upd) => {
    console.log('Occupancy:', upd.occupancy);
  });
  
  space.on('speakerRequest', async (req) => {
    await space.approveSpeaker(req.userId, req.sessionUUID);
  });

  // 7. Cleanup on exit
  process.on('SIGINT', async () => {
    await space.stop();
    process.exit(0);
  });
}

Testing

Run the example:

# 1. Install dependencies
npm install
npm install openai

# 2. Set up environment variables
```bash
TWITTER_USERNAME=YOUR_USERNAME
TWITTER_PASSWORD=YOUR_PASSWORD
TWITTER_EMAIL=YOUR_EMAIL

OPENAI_API_KEY=YOUR_OPENAI_KEY
ELEVENLABS_API_KEY=YOUR_ELEVENLABS_KEY

# 3. Run test
ts-node src/spaces/test.ts

Breaking Changes

None. All new functionality is additive.

Future Work

1. Add more plugin examples (noise reduction, voice effects, Realtime API)
2. Add voice activity detection
3. Add unit tests
4. Add comprehensive documentation

Contact

X: @0xSLK
Discord: slkz.
Telegram: @slkzgm

[odilitime]

Hax! Cracked for sure

[Freytes]

@lalalune its Cracked.

[Bolmstead]

Let's go!! Can't wait to see what's to come

[affaan-m]

Very cool