TGLoader

A powerful Telegram bot and web interface for downloading media from various platforms including YouTube, Instagram, and direct links. Built with Python, Pyrogram, FastAPI, and SQLAlchemy.

Features

• Multi-platform Support: Download videos and media from YouTube, Instagram, and direct URLs
• Telegram Bot Interface: Easy-to-use bot commands for downloading media
• Web Admin Panel: Full-featured web dashboard for task management and user administration
• Task Queue System: Efficient background task processing with progress tracking
• File Management: Automatic file splitting for large files (>2GB), thumbnail generation, and cleanup
• Database Migrations: Automatic database schema migrations using Alembic
• Session Management: Support for both in-memory and Redis-based sessions
• Access Control: Configurable user authorization, admin roles, and private mode
• Cookie Support: Instagram authentication via cookies file
• Rate Limiting: Configurable download limits (file size, duration, concurrent tasks)

Requirements

• Python 3.11+
• PostgreSQL (recommended) or SQLite
• Redis (optional, for session storage)
• FFmpeg (for video thumbnail generation)
• Telegram Bot Token and API credentials

Installation

1. Clone the repository

git clone <repository-url>
cd tgloader

2. Install dependencies

pip install -r requirements.txt

3. Install FFmpeg

Linux:

sudo apt-get install ffmpeg

macOS:

brew install ffmpeg

Windows: Download from ffmpeg.org

4. Configure environment

Create a .env.local file (for local development) or .env file (for Docker):

# Telegram Bot
BOT_TOKEN=your_bot_token
TELEGRAM_API_ID=your_api_id
TELEGRAM_API_HASH=your_api_hash
OWNER_ID=your_telegram_user_id

# Authorization
AUTHORIZED_USERS=user_id1,user_id2
ADMIN_IDS=admin_id1,admin_id2
BLOCKED_USERS=
PRIVATE_MODE=false

# Database
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/tgloader
# Or for SQLite: sqlite+aiosqlite:///./data/bot.db

# Redis (optional)
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0
USE_REDIS_SESSIONS=false

# Web
WEB_HOST=0.0.0.0
WEB_PORT=5000
WEB_SECRET_KEY=your_secret_key_here

# Logging
LOG_LEVEL=INFO
LOG_FILE=logs/bot.log

# Media Download
COOKIES_FILE=instagram.cookies  # Path to cookies file for Instagram (Netscape format)

# Download Limits
MAX_FILE_SIZE=  # Maximum file size in bytes (empty = no limit)
MAX_DURATION_MINUTES=  # Maximum duration in minutes (empty = no limit)
MAX_CONCURRENT_TASKS=5  # Maximum concurrent tasks per user

5. Set up Instagram cookies (optional)

For Instagram downloads, create a instagram.cookies file in Netscape format. You can export cookies from your browser using extensions like "Get cookies.txt LOCALLY" or "cookies.txt".

6. Initialize database

The database will be automatically initialized and migrated on first run using Alembic.

Usage

Running Locally

python main.py

The bot and web interface will start together. Access the web admin panel at http://localhost:5000.

Running with Docker

1. Build and start services:

docker-compose up -d

2. View logs:

docker-compose logs -f app

3. Stop services:

docker-compose down

Telegram Bot Commands

• /start - Start the bot and get help
• /help - Show available commands
• /download <url> - Download media from a URL
• /status - Check your active tasks
• /cancel <task_id> - Cancel a task

Simply send a URL to the bot to start downloading.

Web Admin Panel

Access the admin panel at http://localhost:5000 (or your configured host/port).

Features

• Dashboard: Overview of tasks, users, and system status
• Task Management: View, create, and manage download tasks
• User Management: View and manage authorized users
• OTP Authentication: Secure login with one-time passwords

First Login

1. Enter your Telegram User ID
2. Request an OTP code
3. Check your Telegram messages for the OTP
4. Enter the OTP to log in

Project Structure

tgloader/
├── alembic/              # Database migrations
│   ├── versions/        # Migration files
│   └── env.py           # Alembic environment configuration
├── bot/                 # Telegram bot module
│   ├── modules/
│   │   ├── access_control/  # User authorization and permissions
│   │   ├── media_loader/    # Media download handlers
│   │   ├── message_handler/ # Bot command and callback handlers
│   │   └── task_scheduler/  # Task queue and executor
│   └── utils/           # Utility functions
├── shared/              # Shared code between bot and web
│   ├── config.py       # Application settings
│   └── database/       # Database models and session management
├── web/                 # Web application
│   ├── admin/          # Admin panel routes and templates
│   └── utils/          # Web utilities (auth, CSRF, etc.)
├── main.py             # Application entry point
├── requirements.txt    # Python dependencies
├── docker-compose.yml  # Docker Compose configuration
└── Dockerfile         # Docker image definition

Database Migrations

The project uses Alembic for database migrations. Migrations are automatically applied on startup.

Manual Migration Commands

# Create a new migration
alembic revision --autogenerate -m "Description"

# Apply migrations
alembic upgrade head

# Rollback one migration
alembic downgrade -1

Configuration

Environment Variables

All configuration is done through environment variables. See the .env.local example above for all available options.

Key Settings

• PRIVATE_MODE: If true, only users in AUTHORIZED_USERS or database can use the bot
• USE_REDIS_SESSIONS: Use Redis for session storage instead of in-memory (recommended for production)
• MAX_CONCURRENT_TASKS: Limit the number of simultaneous downloads per user
• COOKIES_FILE: Path to cookies file for Instagram authentication

Development

Code Style

• Follow PEP 8 style guidelines
• Use type hints for all function parameters and return values
• Write docstrings for all classes and functions
• Use English for all code, comments, and documentation

Testing

Run the application locally and test with various URLs:

python main.py

Logging

Logs are written to logs/bot.log by default. Log level can be configured via LOG_LEVEL environment variable.

Troubleshooting

Instagram Downloads Fail

• Ensure COOKIES_FILE is set and the cookies file exists
• Cookies file must be in Netscape format
• Verify cookies are not expired

Database Connection Issues

• Check DATABASE_URL is correct
• Ensure PostgreSQL is running (if using PostgreSQL)
• Verify database credentials

Large File Downloads

Files larger than 2GB are automatically split into parts. Each part is sent separately to Telegram.

License

[Add your license here]

Contributing

[Add contribution guidelines here]

Support

[Add support information here]