# TGLoader

A powerful Telegram bot and web interface for downloading media from various platforms including YouTube, Instagram, and direct links. Built with Python, Pyrogram, FastAPI, and SQLAlchemy.

## Features

- **Multi-platform Support**: Download videos and media from YouTube, Instagram, and direct URLs
- **Telegram Bot Interface**: Easy-to-use bot commands for downloading media
- **Web Admin Panel**: Full-featured web dashboard for task management and user administration
- **Task Queue System**: Efficient background task processing with progress tracking
- **File Management**: Automatic file splitting for large files (>2GB), thumbnail generation, and cleanup
- **Database Migrations**: Automatic database schema migrations using Alembic
- **Session Management**: Support for both in-memory and Redis-based sessions
- **Access Control**: Configurable user authorization, admin roles, and private mode
- **Cookie Support**: Instagram authentication via cookies file
- **Rate Limiting**: Configurable download limits (file size, duration, concurrent tasks)

## Requirements

- Python 3.11+
- PostgreSQL (recommended) or SQLite
- Redis (optional, for session storage)
- FFmpeg (for video thumbnail generation)
- Telegram Bot Token and API credentials

## Installation

### 1. Clone the repository

```bash
git clone <repository-url>
cd tgloader
```

### 2. Install dependencies

```bash
pip install -r requirements.txt
```

### 3. Install FFmpeg

**Linux:**
```bash
sudo apt-get install ffmpeg
```

**macOS:**
```bash
brew install ffmpeg
```

**Windows:**
Download from [ffmpeg.org](https://ffmpeg.org/download.html)

### 4. Configure environment

Create a `.env.local` file (for local development) or `.env` file (for Docker):

```env
# Telegram Bot
BOT_TOKEN=your_bot_token
TELEGRAM_API_ID=your_api_id
TELEGRAM_API_HASH=your_api_hash
OWNER_ID=your_telegram_user_id

# Authorization
AUTHORIZED_USERS=user_id1,user_id2
ADMIN_IDS=admin_id1,admin_id2
BLOCKED_USERS=
PRIVATE_MODE=false

# Database
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/tgloader
# Or for SQLite: sqlite+aiosqlite:///./data/bot.db

# Redis (optional)
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=0
USE_REDIS_SESSIONS=false

# Web
WEB_HOST=0.0.0.0
WEB_PORT=5000
WEB_SECRET_KEY=your_secret_key_here

# Logging
LOG_LEVEL=INFO
LOG_FILE=logs/bot.log

# Media Download
COOKIES_FILE=instagram.cookies  # Path to cookies file for Instagram (Netscape format)

# Download Limits
MAX_FILE_SIZE=  # Maximum file size in bytes (empty = no limit)
MAX_DURATION_MINUTES=  # Maximum duration in minutes (empty = no limit)
MAX_CONCURRENT_TASKS=5  # Maximum concurrent tasks per user
```

### 5. Set up Instagram cookies (optional)

For Instagram downloads, create a `instagram.cookies` file in Netscape format. You can export cookies from your browser using extensions like "Get cookies.txt LOCALLY" or "cookies.txt".

### 6. Initialize database

The database will be automatically initialized and migrated on first run using Alembic.

## Usage

### Running Locally

```bash
python main.py
```

The bot and web interface will start together. Access the web admin panel at `http://localhost:5000`.

### Running with Docker

1. Build and start services:

```bash
docker-compose up -d
```

2. View logs:

```bash
docker-compose logs -f app
```

3. Stop services:

```bash
docker-compose down
```

## Telegram Bot Commands

- `/start` - Start the bot and get help
- `/help` - Show available commands
- `/download <url>` - Download media from a URL
- `/status` - Check your active tasks
- `/cancel <task_id>` - Cancel a task

Simply send a URL to the bot to start downloading.

## Web Admin Panel

Access the admin panel at `http://localhost:5000` (or your configured host/port).

### Features

- **Dashboard**: Overview of tasks, users, and system status
- **Task Management**: View, create, and manage download tasks
- **User Management**: View and manage authorized users
- **OTP Authentication**: Secure login with one-time passwords

### First Login

1. Enter your Telegram User ID
2. Request an OTP code
3. Check your Telegram messages for the OTP
4. Enter the OTP to log in

## Project Structure

```
tgloader/
├── alembic/              # Database migrations
│   ├── versions/        # Migration files
│   └── env.py           # Alembic environment configuration
├── bot/                 # Telegram bot module
│   ├── modules/
│   │   ├── access_control/  # User authorization and permissions
│   │   ├── media_loader/    # Media download handlers
│   │   ├── message_handler/ # Bot command and callback handlers
│   │   └── task_scheduler/  # Task queue and executor
│   └── utils/           # Utility functions
├── shared/              # Shared code between bot and web
│   ├── config.py       # Application settings
│   └── database/       # Database models and session management
├── web/                 # Web application
│   ├── admin/          # Admin panel routes and templates
│   └── utils/          # Web utilities (auth, CSRF, etc.)
├── main.py             # Application entry point
├── requirements.txt    # Python dependencies
├── docker-compose.yml  # Docker Compose configuration
└── Dockerfile         # Docker image definition
```

## Database Migrations

The project uses Alembic for database migrations. Migrations are automatically applied on startup.

### Manual Migration Commands

```bash
# Create a new migration
alembic revision --autogenerate -m "Description"

# Apply migrations
alembic upgrade head

# Rollback one migration
alembic downgrade -1
```

## Configuration

### Environment Variables

All configuration is done through environment variables. See the `.env.local` example above for all available options.

### Key Settings

- **PRIVATE_MODE**: If `true`, only users in `AUTHORIZED_USERS` or database can use the bot
- **USE_REDIS_SESSIONS**: Use Redis for session storage instead of in-memory (recommended for production)
- **MAX_CONCURRENT_TASKS**: Limit the number of simultaneous downloads per user
- **COOKIES_FILE**: Path to cookies file for Instagram authentication

## Development

### Code Style

- Follow PEP 8 style guidelines
- Use type hints for all function parameters and return values
- Write docstrings for all classes and functions
- Use English for all code, comments, and documentation

### Testing

Run the application locally and test with various URLs:

```bash
python main.py
```

### Logging

Logs are written to `logs/bot.log` by default. Log level can be configured via `LOG_LEVEL` environment variable.

## Troubleshooting

### Instagram Downloads Fail

- Ensure `COOKIES_FILE` is set and the cookies file exists
- Cookies file must be in Netscape format
- Verify cookies are not expired

### Database Connection Issues

- Check `DATABASE_URL` is correct
- Ensure PostgreSQL is running (if using PostgreSQL)
- Verify database credentials

### Large File Downloads

Files larger than 2GB are automatically split into parts. Each part is sent separately to Telegram.

## License

[Add your license here]

## Contributing

[Add contribution guidelines here]

## Support

[Add support information here]