Installation Guide

Complete setup instructions for Parliament Scraper on your local development environment.

📋 Requirements

System Requirements

PHP: 8.3+ (recommended) or 8.1+
Laravel: 12+ (current) or 10+
Database: SQLite, MySQL, or PostgreSQL
Node.js: 16+ for asset compilation
Composer: Latest version
FFmpeg: For video processing (optional but recommended)

Optional Requirements

Python: 3.8+ for LangExtract protocol processing
ElevenLabs API Key: For video transcription features
Git: For version control

🚀 Quick Installation

1. Clone the Repository

git clone https://github.com/yourusername/parliament-scraper.git
cd parliament-scraper

2. Install PHP Dependencies

composer install

3. Install Node.js Dependencies

npm install

4. Environment Setup

# Copy environment file
cp .env.example .env

# Generate application key
php artisan key:generate

5. Database Setup

# Run migrations
php artisan migrate

6. Build Assets (Optional)

# For development
npm run dev

# For production
npm run build

⚙️ Environment Configuration

Basic Configuration

Edit your .env file with the following settings:

APP_NAME="Parliament Scraper"
APP_ENV=local
APP_KEY=base64:YOUR_GENERATED_KEY
APP_DEBUG=true
APP_URL=http://localhost

LOG_CHANNEL=stack
LOG_DEPRECATIONS_CHANNEL=null
LOG_LEVEL=debug

# Database Configuration
DB_CONNECTION=sqlite
# DB_HOST=127.0.0.1
# DB_PORT=3306
# DB_DATABASE=parliament_scraper
# DB_USERNAME=root
# DB_PASSWORD=

Database Options

SQLite (Recommended for Development)

DB_CONNECTION=sqlite
# No additional configuration needed

MySQL

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=parliament_scraper
DB_USERNAME=your_username
DB_PASSWORD=your_password

PostgreSQL

DB_CONNECTION=pgsql
DB_HOST=127.0.0.1
DB_PORT=5432
DB_DATABASE=parliament_scraper
DB_USERNAME=your_username
DB_PASSWORD=your_password

🔑 API Keys Configuration

ElevenLabs API (for Video Transcription)

Sign up at ElevenLabs
Get your API key from the dashboard
Add to your .env file:

ELEVENLABS_API_KEY=your_elevenlabs_api_key

OpenAI API (for AI Analysis)

OPENAI_API_KEY=your_openai_api_key

Anthropic API (Alternative AI Provider)

ANTHROPIC_API_KEY=your_anthropic_api_key

Google Gemini API

GEMINI_API_KEY=your_gemini_api_key

🐍 Python Setup (Optional)

For advanced protocol extraction features:

1. Install Python Dependencies

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Configure Python Path

Add to your .env file:

PYTHON_PATH=/path/to/your/python3
# Or if using virtual environment:
PYTHON_PATH=/path/to/your/venv/bin/python

3. Test Installation

php artisan transcripts:extract --check-deps

🎥 FFmpeg Setup (Optional)

For video processing capabilities:

macOS (using Homebrew)

brew install ffmpeg

Ubuntu/Debian

sudo apt update
sudo apt install ffmpeg

Windows

Download from FFmpeg website and add to PATH.

Verify Installation

ffmpeg -version

🗄️ Database Setup Details

Run Migrations

php artisan migrate

Available Tables

The migration will create these tables:

parliament_members - Parliament member information
committees - Committee details
committee_parliament_member - Many-to-many relationships
bills - Legislative bills data
transcripts - Meeting transcripts
bill_analyses - AI analysis results
video_transcriptions - Video transcription data
protocol_extractions - Extracted protocol data

Seed Data (Optional)

# If seeders are available
php artisan db:seed

🔧 Advanced Configuration

File Permissions

Ensure proper permissions for storage and cache directories:

chmod -R 755 storage
chmod -R 755 bootstrap/cache

Symbolic Link for Storage

php artisan storage:link

Cache Configuration

# Clear all caches
php artisan cache:clear
php artisan config:clear
php artisan route:clear
php artisan view:clear

# Optimize for production
php artisan config:cache
php artisan route:cache
php artisan view:cache

🏃‍♂️ Running the Application

Development Server

php artisan serve

Visit http://localhost:8000 in your browser.

Filament Admin Panel

If using Filament admin features:

# Create admin user
php artisan make:filament-user

Visit http://localhost:8000/admin for the admin interface.

✅ Verify Installation

Test Basic Commands

# Test parliament member scraping
php artisan parliament:scrape --help

# Test committee scraping
php artisan committees:scrape --help

# Check available commands
php artisan list

Test Database Connection

# Test with tinker
php artisan tinker
# In tinker: DB::connection()->getPdo()

Test API Access

# Test parliament.bg API access
php artisan committees:scrape --limit=1

🐛 Troubleshooting

Common Issues

Memory Limit Errors

# Increase memory limit for specific commands
php -d memory_limit=-1 artisan your:command

Permission Errors

# Fix storage permissions
sudo chown -R www-data:www-data storage/
sudo chmod -R 755 storage/

Database Connection Issues

Verify database credentials in .env
Ensure database server is running
Check firewall settings

FFmpeg Not Found

# Check if FFmpeg is installed
which ffmpeg

# Install if missing (see FFmpeg setup above)

Laravel-Specific Issues

Key Not Set Error

php artisan key:generate

Migration Errors

# Reset and re-run migrations
php artisan migrate:reset
php artisan migrate

Composer Issues

# Clear composer cache
composer clear-cache
composer install --no-cache

🔄 Updating

Update Dependencies

# Update PHP dependencies
composer update

# Update Node.js dependencies
npm update

# Run any new migrations
php artisan migrate

Update Configuration

# Clear and recache configuration
php artisan config:clear
php artisan config:cache

🚀 Production Deployment

Environment Settings

APP_ENV=production
APP_DEBUG=false
APP_URL=https://yourdomain.com

Optimization

# Install production dependencies
composer install --optimize-autoloader --no-dev

# Cache configuration
php artisan config:cache
php artisan route:cache
php artisan view:cache

# Build production assets
npm run build

Security

Use HTTPS in production
Set strong database passwords
Keep API keys secure
Regular security updates

📞 Support

If you encounter issues during installation:

Check the Troubleshooting section
Review Laravel documentation
Check project issues on GitHub
Ensure all requirements are met

Useful Commands for Debugging

# Check PHP version and extensions
php -v
php -m

# Check Laravel installation
php artisan --version

# View detailed error logs
tail -f storage/logs/laravel.log

# Test database connection
php artisan migrate:status