
GitLab: The All-in-One DevOps Platform
Pro Tip: Use ctrl+F to find text through the article
Table of Contents
- Introduction
- Key Features at a Glance
- The GitLab Philosophy: Who Is It For?
- GitHub vs. GitLab: A Detailed Comparison
- Pros and Cons
- GitLab Installation Guide
- Getting Started: Your First Project
- Mastering GitLab CI/CD
- Advanced DevSecOps Integration
- Project Management & Collaboration
- GitLab Performance & Scaling
- Migration Strategies
- Best Practices & Real-World Examples
- Troubleshooting Common Issues
- Frequently Asked Questions
- Conclusion
Introduction
When discussing alternatives to GitHub, it’s impossible not to start with GitLab. Originally created as an open-source project to provide a self-hosted Git solution, GitLab has evolved into a comprehensive, single-application platform for the entire DevOps lifecycle. Its core mission is to streamline the software development process by eliminating the need for a complex toolchain of disparate applications.
Instead of stitching together tools for planning, source code management, CI/CD, and security, GitLab offers a cohesive, integrated experience from idea to production.
Why This Guide Matters:
- Over 30 million registered users trust GitLab for their DevOps needs
- Companies report 200%+ efficiency gains when consolidating to GitLab
- GitLab handles over 1 billion CI/CD jobs annually
- 50% of Fortune 100 companies use GitLab in some capacity
Key Features at a Glance
GitLab’s power lies in its breadth of features. While it excels at source code management, its true value is in the integrated components that support every stage of development.
Feature Comparison Matrix
Feature Category | GitLab Community (Free) | GitLab Premium | GitLab Ultimate | Enterprise Impact |
---|---|---|---|---|
Source Code Management | ✅ Full Git functionality | ✅ + Advanced merge requests | ✅ + Push rules & compliance | Foundation for all development |
CI/CD Pipelines | ✅ Unlimited private repos | ✅ + Multiple approval rules | ✅ + Security scanning | 3x faster deployment cycles |
Security & Compliance | ✅ Basic dependency scanning | ✅ + SAST/DAST scanning | ✅ + Compliance management | 85% reduction in vulnerabilities |
Project Management | ✅ Issues & milestones | ✅ + Roadmaps & epics | ✅ + Portfolio management | 40% improvement in delivery predictability |
Monitoring & Analytics | ✅ Basic usage analytics | ✅ + Advanced analytics | ✅ + Value stream analytics | Data-driven decision making |
Core Feature Deep Dive
Feature | Description | Key Benefit | Business Value |
---|---|---|---|
Source Code Management | Robust Git repository hosting with merge requests, advanced code review, protected branches, and a web-based IDE. | Provides a powerful and familiar Git workflow at its core, comparable to the best in the industry. | Faster code reviews, reduced bugs |
Integrated CI/CD | A market-leading Continuous Integration/Delivery/Deployment solution built directly into the platform, configured via a .gitlab-ci.yml file. | Eliminates the need for third-party CI tools like Jenkins. Pipelines are version-controlled with your code, enabling seamless automation. | 50% reduction in deployment time |
Integrated Security (DevSecOps) | A suite of security tools including SAST, DAST, dependency scanning, and container scanning are built into the CI/CD pipeline. | ”Shifts security left” by finding vulnerabilities early in the development process, directly within the developer’s workflow. | 70% fewer production security issues |
Advanced Project Management | Includes issue tracking, epics, roadmaps, Kanban boards, and time tracking to manage projects from planning to release. | Offers a single source of truth for project planning and tracking, potentially replacing tools like Jira or Trello. | 30% improvement in project visibility |
The GitLab Philosophy: Who Is It For?
GitLab is built for teams and enterprises that want to adopt a true DevOps culture by consolidating their toolchain. Its philosophy is centered on a “single application” approach.
Target Audience Analysis
Enterprises (1000+ employees)
- Primary Need: Standardization and governance at scale
- Key Benefits: Compliance frameworks, advanced analytics, enterprise support
- ROI Timeline: 6-12 months
- Typical Use Case: Large-scale software delivery with multiple teams and complex compliance requirements
Mid-size Companies (100-1000 employees)
- Primary Need: Efficient toolchain consolidation without enterprise complexity
- Key Benefits: Integrated CI/CD, simplified workflows, cost reduction
- ROI Timeline: 3-6 months
- Typical Use Case: Growing engineering teams looking to scale development practices
Startups & Small Teams (10-100 employees)
- Primary Need: Maximum functionality with minimal overhead
- Key Benefits: Free tier capabilities, self-hosting options, all-in-one platform
- ROI Timeline: Immediate
- Typical Use Case: Resource-constrained teams wanting enterprise-grade tools
Open Source Projects
- Primary Need: Community collaboration and transparency
- Key Benefits: Public repositories, community features, free hosting
- ROI Timeline: Immediate
- Typical Use Case: Open source maintainers seeking professional development tools
This makes it the ideal choice for:
Teams valuing self-hosting: GitLab’s free and open-source Community Edition offers unparalleled power for teams who want full control over their infrastructure and data.
DevSecOps Adopters: Organizations that prioritize security and want to integrate it seamlessly into their development lifecycle without adding friction.
If your team is tired of “integration hell” and wants a single, cohesive platform to manage everything from planning and coding to building, securing, and deploying applications, GitLab is designed for you.
GitHub vs. GitLab: A Detailed Comparison
While both platforms are leaders in Git hosting, their core focus and approach differ significantly.
Feature-by-Feature Comparison
Aspect | GitHub | GitLab | Winner |
---|---|---|---|
Primary Focus | Community, Open Source, and Developer Experience | All-in-one DevOps Platform, End-to-End Workflow | Tie |
Hosting Options | Cloud (SaaS) and Self-hosted (Enterprise only) | Cloud (SaaS) and Self-hosted (Free Community & Paid Enterprise) | GitLab |
CI/CD | GitHub Actions (Extensible marketplace) | Built-in GitLab CI/CD (Tightly integrated) | GitLab |
DevOps Scope | Core SCM with integrated CI/CD and project management. Relies on marketplace for many other stages. | A single application covering the entire DevOps lifecycle, including planning, security, and monitoring. | GitLab |
Community Size | 100M+ developers, largest open source community | 30M+ registered users, growing enterprise adoption | GitHub |
Pricing Model | Free public repos, paid private repos | Free private repos (up to limits), tiered pricing | GitLab |
Security Features | Dependabot, secret scanning (paid tiers) | Comprehensive built-in security suite | GitLab |
Project Management | Issues, projects, limited planning tools | Full project management suite with epics, roadmaps | GitLab |
Performance Metrics Comparison
Metric | GitHub | GitLab | Industry Impact |
---|---|---|---|
Repository Size Limit | 100GB (LFS) | Unlimited (self-hosted) | Critical for large codebases |
CI/CD Build Minutes | 2,000/month (free) | 400/month (free) | Affects development velocity |
Concurrent Jobs | 20 (free) | Shared runners | Pipeline efficiency |
API Rate Limiting | 5,000/hour | 600/minute | Integration capabilities |
Uptime SLA | 99.95% | 99.95% | Business continuity |
Pros and Cons
Why You Might Choose GitLab
True All-in-One Platform
The biggest advantage. A single, unified UI and data model for your entire workflow simplifies processes and improves team collaboration.
Quantified Benefits:
- 40% reduction in context switching between tools
- 25% faster onboarding for new team members
- 60% improvement in cross-team visibility
Superior Self-Hosting Flexibility
GitLab’s Community Edition is a feature-rich, free, and open-source option for self-hosting, giving you complete control and data privacy.
Self-Hosting Advantages:
- Complete data sovereignty
- Custom compliance requirements
- Unlimited storage and compute resources
- No vendor lock-in concerns
Powerful Integrated CI/CD
GitLab CI/CD is mature, robust, and tightly integrated. The Auto DevOps feature can automatically build, test, and deploy applications with zero configuration.
CI/CD Metrics:
- 3x faster pipeline setup compared to Jenkins
- 50% reduction in build failures due to configuration errors
- Built-in Docker registry and Kubernetes integration
Built-in Security Scanning (DevSecOps)
Integrating security scanning directly into merge requests is a game-changer for building secure applications from the start.
Security Impact:
- 70% of vulnerabilities caught before production
- 90% reduction in security review cycle time
- Automated compliance reporting
Potential Drawbacks
Can Feel Overwhelming
The sheer number of features can make the UI feel cluttered and complex compared to GitHub’s more focused interface.
Mitigation Strategies:
- Use GitLab’s role-based access to hide unused features
- Implement phased rollout starting with core SCM features
- Provide comprehensive team training
Resource-Intensive Self-Hosting
Running the full GitLab omnibus package requires a significant amount of server resources (RAM and CPU), which can be a barrier for smaller teams or individuals.
Resource Requirements:
- Minimum: 4GB RAM, 2 CPU cores
- Recommended: 8GB RAM, 4 CPU cores
- Enterprise: 16GB+ RAM, 8+ CPU cores
Slightly Steeper Learning Curve
Because it does so much, mastering the entire platform takes more time than learning a more focused tool.
Learning Path Recommendations:
- Week 1-2: Source code management and basic workflows
- Week 3-4: CI/CD pipeline configuration
- Month 2: Advanced features and integrations
- Month 3+: DevSecOps and enterprise features
GitLab Installation Guide
Cloud vs. Self-Hosted Decision Matrix
Factor | GitLab.com (SaaS) | Self-Hosted | Recommendation |
---|---|---|---|
Setup Time | Immediate | 1-4 hours | SaaS for quick start |
Maintenance | Zero | High | SaaS for small teams |
Customization | Limited | Full control | Self-hosted for enterprises |
Data Privacy | Shared infrastructure | Complete control | Self-hosted for sensitive data |
Scalability | Automatic | Manual planning | SaaS for unpredictable growth |
Cost (100 users) | $19/user/month | Server costs + admin time | Varies by requirements |
Self-Hosted Installation: Step-by-Step Guide
Prerequisites Checklist
# System requirements verification script
#!/bin/bash
echo "GitLab Installation Prerequisites Check"
echo "======================================"
# Check OS version
echo "OS Version:"
cat /etc/os-release
# Check available memory
echo -e "\nMemory Check:"
free -h
# Check available disk space
echo -e "\nDisk Space Check:"
df -h
# Check if Docker is installed (for Docker installation)
echo -e "\nDocker Status:"
docker --version 2>/dev/null || echo "Docker not installed"
# Check if required ports are available
echo -e "\nPort Availability Check:"
for port in 80 443 22; do
netstat -tuln | grep ":$port " && echo "Port $port is in use" || echo "Port $port is available"
done
Installation Method 1: Ubuntu/Debian Package Installation
# Step 1: Install dependencies
sudo apt-get update
sudo apt-get install -y curl openssh-server ca-certificates tzdata perl
# Step 2: Configure firewall (if using ufw)
sudo ufw allow OpenSSH
sudo ufw allow http
sudo ufw allow https
sudo ufw enable
# Step 3: Add GitLab package repository
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
# Step 4: Install GitLab Community Edition
sudo EXTERNAL_URL="https://gitlab.example.com" apt-get install gitlab-ce
# Step 5: Initial configuration
sudo gitlab-ctl reconfigure
Installation Method 2: Docker Compose Installation
Create a docker-compose.yml
file:
version: "3.8"
services:
gitlab:
image: "gitlab/gitlab-ce:latest"
hostname: "gitlab.example.com"
container_name: gitlab
restart: always
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://gitlab.example.com'
gitlab_rails['gitlab_shell_ssh_port'] = 2224
# Email configuration
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.gmail.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "[email protected]"
gitlab_rails['smtp_password'] = "your-app-password"
gitlab_rails['smtp_domain'] = "smtp.gmail.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = false
ports:
- "80:80"
- "443:443"
- "2224:22"
volumes:
- "$GITLAB_HOME/config:/etc/gitlab"
- "$GITLAB_HOME/logs:/var/log/gitlab"
- "$GITLAB_HOME/data:/var/opt/gitlab"
shm_size: "256m"
# Set environment variable for data location
export GITLAB_HOME=/srv/gitlab
# Create directories
sudo mkdir -p $GITLAB_HOME/{config,logs,data}
# Start GitLab
docker-compose up -d
# Monitor startup progress
docker logs -f gitlab
Installation Method 3: Kubernetes Helm Chart
# Step 1: Add GitLab Helm repository
helm repo add gitlab https://charts.gitlab.io/
helm repo update
# Step 2: Create values file
cat <<EOF > gitlab-values.yaml
global:
hosts:
domain: example.com
ingress:
configureCertmanager: true
email:
from: '[email protected]'
display_name: GitLab
reply_to: '[email protected]'
certmanager:
install: true
postgresql:
install: true
redis:
install: true
registry:
enabled: true
gitlab-runner:
install: true
EOF
# Step 3: Install GitLab
kubectl create namespace gitlab
helm install gitlab gitlab/gitlab \
--namespace gitlab \
--values gitlab-values.yaml \
--timeout 600s
Post-Installation Configuration
Initial Admin Setup
# Get initial root password (for package installations)
sudo cat /etc/gitlab/initial_root_password
# For Docker installations
docker exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password
SSL Certificate Configuration
# Using Let's Encrypt (recommended for production)
sudo gitlab-ctl reconfigure
# Manual SSL certificate configuration
sudo mkdir -p /etc/gitlab/ssl
sudo cp your-domain.crt /etc/gitlab/ssl/
sudo cp your-domain.key /etc/gitlab/ssl/
sudo chmod 600 /etc/gitlab/ssl/*
SMTP Configuration for Email Notifications
Edit /etc/gitlab/gitlab.rb
:
# SMTP Settings
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.office365.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "[email protected]"
gitlab_rails['smtp_password'] = "your-password"
gitlab_rails['smtp_domain'] = "yourcompany.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = false
# GitLab email sender
gitlab_rails['gitlab_email_from'] = '[email protected]'
gitlab_rails['gitlab_email_display_name'] = 'GitLab'
Apply configuration:
sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart
# Test email configuration
sudo gitlab-rails console
> Notify.test_email('[email protected]', 'Test Subject', 'Test Body').deliver_now
Backup Configuration
# Configure automated backups
cat <<EOF >> /etc/gitlab/gitlab.rb
# Backup settings
gitlab_rails['backup_keep_time'] = 604800 # 7 days
gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"
# Upload backups to S3 (optional)
gitlab_rails['backup_upload_connection'] = {
'provider' => 'AWS',
'region' => 'us-east-1',
'aws_access_key_id' => 'your-access-key',
'aws_secret_access_key' => 'your-secret-key'
}
gitlab_rails['backup_upload_remote_directory'] = 'your-backup-bucket'
EOF
# Reconfigure and test backup
sudo gitlab-ctl reconfigure
sudo gitlab-backup create
Runner Registration
# Install GitLab Runner
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
sudo apt-get install gitlab-runner
# Register runner (get token from GitLab admin area)
sudo gitlab-runner register \
--url "https://gitlab.example.com/" \
--registration-token "your-registration-token" \
--description "docker-runner" \
--executor "docker" \
--docker-image alpine:latest
Getting Started: Your First Project
Project Creation Walkthrough
1. Creating a New Project
# Via GitLab UI: Projects > New Project > Create blank project
# Via GitLab CLI (glab)
glab repo create my-first-project --private --description "My first GitLab project"
# Via API
curl --request POST --header "PRIVATE-TOKEN: your-token" \
--header "Content-Type: application/json" \
--data '{"name":"my-first-project","description":"My first GitLab project","visibility":"private"}' \
"https://gitlab.example.com/api/v4/projects"
2. Repository Setup
# Clone your new repository
git clone https://gitlab.example.com/username/my-first-project.git
cd my-first-project
# Create initial project structure
mkdir -p {src,tests,docs,scripts}
touch README.md .gitignore
# Create a sample application
cat <<EOF > src/app.py
#!/usr/bin/env python3
"""
Sample Python application for GitLab demonstration
"""
def hello_world(name: str = "World") -> str:
"""Return a greeting message."""
return f"Hello, {name}!"
def main():
"""Main application entry point."""
print(hello_world("GitLab"))
if __name__ == "__main__":
main()
EOF
# Create requirements file
cat <<EOF > requirements.txt
pytest>=7.0.0
pytest-cov>=4.0.0
flake8>=5.0.0
EOF
# Create test file
cat <<EOF > tests/test_app.py
"""Tests for the sample application."""
import pytest
from src.app import hello_world
def test_hello_world_default():
"""Test hello_world with default parameter."""
result = hello_world()
assert result == "Hello, World!"
def test_hello_world_custom_name():
"""Test hello_world with custom name."""
result = hello_world("GitLab")
assert result == "Hello, GitLab!"
EOF
3. GitLab Configuration Files
Create .gitlab-ci.yml
for CI/CD:
# GitLab CI/CD Pipeline Configuration
stages:
- lint
- test
- build
- security
- deploy
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
cache:
paths:
- .cache/pip/
- venv/
before_script:
- python -V # Print Python version for debugging
- pip install virtualenv
- virtualenv venv
- source venv/bin/activate
- pip install -r requirements.txt
# Linting stage
lint:
stage: lint
script:
- flake8 src/ tests/
only:
- merge_requests
- main
# Testing stage
test:
stage: test
script:
- pytest tests/ --cov=src --cov-report=xml --cov-report=term
coverage: "/TOTAL.+ ([0-9]{1,3}%)/"
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
only:
- merge_requests
- main
# Security scanning
security_scan:
stage: security
script:
- echo "Running security scans..."
- pip install safety bandit
- safety check
- bandit -r src/
only:
- merge_requests
- main
# Build stage
build:
stage: build
script:
- echo "Building application..."
- python setup.py sdist bdist_wheel
artifacts:
paths:
- dist/
only:
- main
# Deploy to staging
deploy_staging:
stage: deploy
script:
- echo "Deploying to staging environment..."
- echo "Application URL: https://staging.example.com"
environment:
name: staging
url: https://staging.example.com
only:
- main
# Deploy to production (manual trigger)
deploy_production:
stage: deploy
script:
- echo "Deploying to production environment..."
- echo "Application URL: https://production.example.com"
environment:
name: production
url: https://production.example.com
when: manual
only:
- main
Create project setup file:
# setup.py
from setuptools import setup, find_packages
setup(
name="my-first-project",
version="0.1.0",
packages=find_packages(),
install_requires=[],
author="Your Name",
author_email="[email protected]",
description="A sample GitLab project",
long_description=open("README.md").read(),
long_description_content_type="text/markdown",
url="https://gitlab.example.com/username/my-first-project",
classifiers=[
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
],
python_requires=">=3.8",
)
4. Initial Commit and Push
# Configure Git (if not already done)
git config user.name "Your Name"
git config user.email "[email protected]"
# Add files and commit
git add .
git commit -m "Initial commit: Add sample Python application with CI/CD pipeline"
# Push to GitLab
git push -u origin main
Setting Up Branch Protection and Merge Requests
1. Configure Protected Branches
Navigate to Project Settings > Repository > Protected branches or use the API:
# Protect main branch via API
curl --request POST --header "PRIVATE-TOKEN: your-token" \
"https://gitlab.example.com/api/v4/projects/PROJECT_ID/protected_branches" \
--form "name=main" \
--form "push_access_level=40" \
--form "merge_access_level=40"
2. Create Your First Merge Request
# Create a feature branch
git checkout -b feature/add-documentation
# Add documentation
cat <<EOF > docs/INSTALLATION.md
# Installation Guide
## Requirements
- Python 3.8 or higher
- pip package manager
## Installation Steps
1. Clone the repository
2. Install dependencies: \`pip install -r requirements.txt\`
3. Run the application: \`python src/app.py\`
## Running Tests
\`\`\`bash
pytest tests/
\`\`\`
EOF
# Commit and push
git add docs/INSTALLATION.md
git commit -m "Add installation documentation"
git push -u origin feature/add-documentation
# Create merge request via glab CLI
glab mr create --title "Add installation documentation" \
--description "This MR adds comprehensive installation documentation for new users"
Mastering GitLab CI/CD
Pipeline Architecture Overview
GitLab CI/CD follows a stage-based architecture where jobs run in parallel within stages and stages run sequentially.
graph LR
A[Source Code] --> B[Build Stage]
B --> C[Test Stage]
C --> D[Security Stage]
D --> E[Deploy Staging]
E --> F[Deploy Production]
B --> B1[Compile]
B --> B2[Package]
C --> C1[Unit Tests]
C --> C2[Integration Tests]
C --> C3[Code Coverage]
D --> D1[SAST Scan]
D --> D2[DAST Scan]
D --> D3[Dependency Scan]
Advanced Pipeline Examples
Multi-Language Monorepo Pipeline
# .gitlab-ci.yml for monorepo with multiple services
stages:
- analyze
- build
- test
- security
- package
- deploy
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
# Template for service jobs
.service_template: &service_template
rules:
- changes:
- "$SERVICE_PATH/**/*"
- ".gitlab-ci.yml"
# Analyze stage - detect changes
detect_changes:
stage: analyze
image: alpine/git
script:
- |
echo "Detecting changes..."
if git diff --name-only $CI_COMMIT_BEFORE_SHA $CI_COMMIT_SHA | grep -q "^frontend/"; then
echo "FRONTEND_CHANGED=true" >> variables.env
fi
if git diff --name-only $CI_COMMIT_BEFORE_SHA $CI_COMMIT_SHA | grep -q "^backend/"; then
echo "BACKEND_CHANGED=true" >> variables.env
fi
if git diff --name-only $CI_COMMIT_BEFORE_SHA $CI_COMMIT_SHA | grep -q "^mobile/"; then
echo "MOBILE_CHANGED=true" >> variables.env
fi
artifacts:
reports:
dotenv: variables.env
# Frontend jobs
frontend_lint:
stage: build
image: node:16-alpine
<<: *service_template
variables:
SERVICE_PATH: frontend
script:
- cd frontend
- npm ci
- npm run lint
- npm run type-check
cache:
key: frontend-$CI_COMMIT_REF_SLUG
paths:
- frontend/node_modules/
frontend_test:
stage: test
image: node:16-alpine
<<: *service_template
variables:
SERVICE_PATH: frontend
script:
- cd frontend
- npm ci
- npm run test:coverage
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: frontend/coverage/cobertura-coverage.xml
cache:
key: frontend-$CI_COMMIT_REF_SLUG
paths:
- frontend/node_modules/
frontend_build:
stage: package
image: node:16-alpine
<<: *service_template
variables:
SERVICE_PATH: frontend
script:
- cd frontend
- npm ci
- npm run build
artifacts:
paths:
- frontend/dist/
expire_in: 1 hour
# Backend jobs
backend_lint:
stage: build
image: python:3.9-slim
<<: *service_template
variables:
SERVICE_PATH: backend
before_script:
- cd backend
- pip install -r requirements-dev.txt
script:
- flake8 .
- black --check .
- mypy .
backend_test:
stage: test
image: python:3.9-slim
<<: *service_template
variables:
SERVICE_PATH: backend
services:
- postgres:13
- redis:6-alpine
variables:
POSTGRES_DB: testdb
POSTGRES_USER: testuser
POSTGRES_PASSWORD: testpass
DATABASE_URL: postgresql://testuser:testpass@postgres:5432/testdb
REDIS_URL: redis://redis:6379
before_script:
- cd backend
- pip install -r requirements.txt
- pip install -r requirements-dev.txt
script:
- pytest --cov=app --cov-report=xml --cov-report=term
coverage: '/TOTAL.+ ([0-9]{1,3}%)/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: backend/coverage.xml
# Container build jobs
.docker_build_template: &docker_build_template
stage: package
image: docker:20.10.16
services:
- docker:20.10.16-dind
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- |
docker build -t $CI_REGISTRY_IMAGE/$SERVICE_NAME:$CI_COMMIT_SHA \
-t $CI_REGISTRY_IMAGE/$SERVICE_NAME:latest \
$SERVICE_PATH/
docker push $CI_REGISTRY_IMAGE/$SERVICE_NAME:$CI_COMMIT_SHA
docker push $CI_REGISTRY_IMAGE/$SERVICE_NAME:latest
frontend_docker:
<<: *docker_build_template
variables:
SERVICE_NAME: frontend
SERVICE_PATH: frontend
rules:
- changes:
- "frontend/**/*"
- ".gitlab-ci.yml"
backend_docker:
<<: *docker_build_template
variables:
SERVICE_NAME: backend
SERVICE_PATH: backend
rules:
- changes:
- "backend/**/*"
- ".gitlab-ci.yml"
# Security scanning
include:
- template: Security/SAST.gitlab-ci.yml
- template: Security/Dependency-Scanning.gitlab-ci.yml
- template: Security/Container-Scanning.gitlab-ci.yml
- template: Security/DAST.gitlab-ci.yml
# Custom security job for secrets detection
secrets_scan:
stage: security
image: alpine/git
script:
- apk add --no-cache git
- git clone https://github.com/trufflesecurity/trufflehog.git
- cd trufflehog && go build
- ./trufflehog filesystem ../
allow_failure: true
# Deployment jobs
.deploy_template: &deploy_template
image: alpine/k8s:1.24.0
before_script:
- echo $KUBECONFIG | base64 -d > /tmp/kubeconfig
- export KUBECONFIG=/tmp/kubeconfig
- kubectl version --client
deploy_staging:
<<: *deploy_template
stage: deploy
environment:
name: staging
url: https://staging.example.com
deployment_tier: staging
script:
- |
helm upgrade --install myapp-staging ./helm-chart \
--namespace staging \
--set image.frontend.tag=$CI_COMMIT_SHA \
--set image.backend.tag=$CI_COMMIT_SHA \
--set ingress.host=staging.example.com
only:
- main
deploy_production:
<<: *deploy_template
stage: deploy
environment:
name: production
url: https://production.example.com
deployment_tier: production
script:
- |
helm upgrade --install myapp-production ./helm-chart \
--namespace production \
--set image.frontend.tag=$CI_COMMIT_SHA \
--set image.backend.tag=$CI_COMMIT_SHA \
--set ingress.host=production.example.com
when: manual
only:
- main
Dynamic Pipeline Configuration
# Dynamic pipeline based on project structure
stages:
- prepare
- build
- test
- deploy
# Job to generate dynamic pipeline
generate_pipeline:
stage: prepare
image: alpine:latest
script:
- |
cat > dynamic-pipeline.yml << EOF
stages:
- build
- test
- deploy
EOF
# Check for different project types
if [ -f "package.json" ]; then
cat >> dynamic-pipeline.yml << EOF
node_build:
stage: build
image: node:16
script:
- npm ci
- npm run build
artifacts:
paths:
- dist/
node_test:
stage: test
image: node:16
script:
- npm ci
- npm test
EOF
fi
if [ -f "requirements.txt" ]; then
cat >> dynamic-pipeline.yml << EOF
python_build:
stage: build
image: python:3.9
script:
- pip install -r requirements.txt
- python setup.py sdist bdist_wheel
artifacts:
paths:
- dist/
python_test:
stage: test
image: python:3.9
script:
- pip install -r requirements.txt
- pytest
EOF
fi
if [ -f "Dockerfile" ]; then
cat >> dynamic-pipeline.yml << EOF
docker_build:
stage: build
image: docker:20.10.16
services:
- docker:20.10.16-dind
script:
- docker build -t \$CI_REGISTRY_IMAGE:\$CI_COMMIT_SHA .
- docker push \$CI_REGISTRY_IMAGE:\$CI_COMMIT_SHA
EOF
fi
artifacts:
paths:
- dynamic-pipeline.yml
rules:
- if: $CI_COMMIT_BRANCH == "main"
# Include the dynamically generated pipeline
include:
- local: dynamic-pipeline.yml
GitLab Runner Configuration
Installing and Configuring Runners
# Install GitLab Runner on Ubuntu
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
sudo apt-get install gitlab-runner
# Create custom config for Docker executor
cat <<EOF > /tmp/runner-config.toml
concurrent = 10
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker-runner-1"
url = "https://gitlab.example.com/"
token = "your-runner-token"
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "alpine:latest"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
shm_size = 0
EOF
# Copy config and restart
sudo cp /tmp/runner-config.toml /etc/gitlab-runner/config.toml
sudo gitlab-runner restart
Kubernetes Runner with Auto-scaling
# kubernetes-runner-values.yaml
gitlabUrl: https://gitlab.example.com/
runnerToken: "your-runner-token"
runners:
config: |
[[runners]]
[runners.kubernetes]
image = "ubuntu:20.04"
privileged = true
namespace = "gitlab-runner"
# Resource limits
cpu_limit = "1"
memory_limit = "2Gi"
service_cpu_limit = "0.5"
service_memory_limit = "512Mi"
# Node selector for dedicated nodes
node_selector_overwrite_allowed = ".*"
# Helper image configuration
helper_cpu_limit = "0.1"
helper_memory_limit = "128Mi"
# Volume mounts
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
# Install with Helm
helm repo add gitlab https://charts.gitlab.io
helm install gitlab-runner gitlab/gitlab-runner \
--namespace gitlab-runner \
--create-namespace \
--values kubernetes-runner-values.yaml
Custom Runner with GPU Support
# GPU runner configuration
[[runners]]
name = "gpu-runner"
url = "https://gitlab.example.com/"
token = "your-gpu-runner-token"
executor = "docker"
[runners.docker]
image = "nvidia/cuda:11.8-runtime-ubuntu20.04"
privileged = true
runtime = "nvidia"
environment = ["NVIDIA_VISIBLE_DEVICES=all"]
volumes = ["/var/run/docker.sock:/var/run/docker.sock"]
Pipeline Optimization Strategies
Caching Best Practices
# Global cache configuration
variables:
CACHE_VERSION: "v1"
# Cache templates for different languages
.python_cache: &python_cache
cache:
key: python-$CACHE_VERSION-$CI_COMMIT_REF_SLUG
paths:
- .pip-cache/
- venv/
policy: pull-push
.node_cache: &node_cache
cache:
key: node-$CACHE_VERSION-$CI_COMMIT_REF_SLUG
paths:
- node_modules/
- .npm/
policy: pull-push
.maven_cache: &maven_cache
cache:
key: maven-$CACHE_VERSION-$CI_COMMIT_REF_SLUG
paths:
- .m2/repository/
policy: pull-push
# Example job with optimized caching
python_test:
stage: test
image: python:3.9
<<: *python_cache
before_script:
- export PIP_CACHE_DIR=.pip-cache
- python -m venv venv
- source venv/bin/activate
- pip install --upgrade pip
- pip install -r requirements.txt
script:
- pytest --junitxml=report.xml --cov=src --cov-report=xml
artifacts:
reports:
junit: report.xml
coverage_report:
coverage_format: cobertura
path: coverage.xml
Parallel Job Execution
# Matrix builds for multiple environments
test:
stage: test
image: python:${PYTHON_VERSION}
parallel:
matrix:
- PYTHON_VERSION: ["3.8", "3.9", "3.10", "3.11"]
OS: ["ubuntu-latest"]
- PYTHON_VERSION: ["3.9"]
OS: ["windows-latest", "macos-latest"]
script:
- pip install -r requirements.txt
- pytest
# Parallel test execution by test suite
test_unit:
stage: test
script:
- pytest tests/unit/
parallel: 4
test_integration:
stage: test
script:
- pytest tests/integration/
parallel: 2
# Fan-out/fan-in pattern for microservices
build_services:
stage: build
script:
- echo "Building service $SERVICE_NAME"
parallel:
matrix:
- SERVICE_NAME: ["auth", "user", "payment", "notification"]
deploy_all:
stage: deploy
needs:
- job: build_services
parallel:
matrix:
- SERVICE_NAME: ["auth", "user", "payment", "notification"]
script:
- echo "Deploying all services"
Environment Management
Environment-specific Deployments
# Environment configuration
.deploy_template: &deploy_template
stage: deploy
image: alpine/k8s:1.24.0
before_script:
- echo $KUBECONFIG | base64 -d > /tmp/kubeconfig
- export KUBECONFIG=/tmp/kubeconfig
# Development environment
deploy_development:
<<: *deploy_template
environment:
name: development
url: https://dev.example.com
deployment_tier: development
auto_stop_in: 1 day
script:
- helm upgrade --install myapp-dev ./chart --set env=development
rules:
- if: $CI_COMMIT_BRANCH != "main"
when: manual
# Staging environment
deploy_staging:
<<: *deploy_template
environment:
name: staging
url: https://staging.example.com
deployment_tier: staging
script:
- helm upgrade --install myapp-staging ./chart --set env=staging
rules:
- if: $CI_COMMIT_BRANCH == "main"
# Production environment with approval
deploy_production:
<<: *deploy_template
environment:
name: production
url: https://production.example.com
deployment_tier: production
script:
- helm upgrade --install myapp-prod ./chart --set env=production
rules:
- if: $CI_COMMIT_BRANCH == "main"
when: manual
allow_failure: false
Blue-Green Deployment Strategy
# Blue-Green deployment implementation
deploy_blue:
stage: deploy
environment:
name: production-blue
url: https://blue.production.example.com
script:
- |
helm upgrade --install myapp-blue ./chart \
--set deployment.color=blue \
--set image.tag=$CI_COMMIT_SHA \
--set ingress.host=blue.production.example.com
- ./scripts/health-check.sh blue.production.example.com
artifacts:
reports:
dotenv: blue-deployment.env
smoke_test_blue:
stage: deploy
needs: ["deploy_blue"]
script:
- ./scripts/smoke-tests.sh blue.production.example.com
- echo "BLUE_HEALTH=healthy" >> deployment.env
artifacts:
reports:
dotenv: deployment.env
switch_traffic:
stage: deploy
needs: ["smoke_test_blue"]
environment:
name: production
url: https://production.example.com
script:
- |
# Update load balancer to point to blue environment
kubectl patch service production-service \
-p '{"spec":{"selector":{"color":"blue"}}}'
- echo "Traffic switched to blue environment"
- ./scripts/monitor-metrics.sh
when: manual
cleanup_green:
stage: deploy
needs: ["switch_traffic"]
script:
- helm uninstall myapp-green || true
- echo "Green environment cleaned up"
when: manual
Advanced DevSecOps Integration
Security Scanning Configuration
SAST (Static Application Security Testing)
# Custom SAST configuration
include:
- template: Security/SAST.gitlab-ci.yml
variables:
SAST_EXCLUDED_PATHS: "tests/, docs/, *.md"
SAST_EXCLUDED_ANALYZERS: "eslint" # Exclude specific analyzers
# Custom SAST job for additional tools
sonarqube_scan:
stage: security
image: sonarsource/sonar-scanner-cli:latest
variables:
SONAR_HOST_URL: "https://sonarqube.example.com"
SONAR_TOKEN: $SONAR_TOKEN
GIT_DEPTH: "0"
script:
- sonar-scanner
-Dsonar.projectKey=$CI_PROJECT_NAME
-Dsonar.sources=src/
-Dsonar.tests=tests/
-Dsonar.python.coverage.reportPaths=coverage.xml
-Dsonar.python.xunit.reportPath=pytest-report.xml
coverage: '/COVERAGE.+?(\d+(?:\.\d+)?%)/'
artifacts:
reports:
junit: pytest-report.xml
# Custom security rules enforcement
security_policy_check:
stage: security
image: alpine:latest
before_script:
- apk add --no-cache jq curl
script:
- |
# Download security scan results
SCAN_RESULTS=$(curl -s --header "PRIVATE-TOKEN: $CI_JOB_TOKEN" \
"$CI_API_V4_URL/projects/$CI_PROJECT_ID/vulnerability_findings")
# Count critical and high vulnerabilities
CRITICAL=$(echo "$SCAN_RESULTS" | jq '[.[] | select(.severity == "Critical")] | length')
HIGH=$(echo "$SCAN_RESULTS" | jq '[.[] | select(.severity == "High")] | length')
echo "Critical vulnerabilities: $CRITICAL"
echo "High vulnerabilities: $HIGH"
# Fail if policy violations exist
if [ "$CRITICAL" -gt 0 ]; then
echo "❌ Policy violation: Critical vulnerabilities found"
exit 1
fi
if [ "$HIGH" -gt 5 ]; then
echo "❌ Policy violation: Too many high-severity vulnerabilities ($HIGH > 5)"
exit 1
fi
echo "✅ Security policy compliance check passed"
DAST (Dynamic Application Security Testing)
# DAST configuration
include:
- template: DAST.gitlab-ci.yml
variables:
DAST_WEBSITE: "https://staging.example.com"
DAST_FULL_SCAN_ENABLED: "true"
DAST_AUTH_URL: "https://staging.example.com/login"
DAST_USERNAME: $DAST_USERNAME
DAST_PASSWORD: $DAST_PASSWORD
DAST_EXCLUDE_URLS: "https://staging.example.com/admin,https://staging.example.com/logout"
# Custom DAST with OWASP ZAP
owasp_zap_scan:
stage: security
image: owasp/zap2docker-stable:latest
variables:
ZAP_TARGET: "https://staging.example.com"
script:
- mkdir -p /zap/wrk
- |
zap-full-scan.py \
-t $ZAP_TARGET \
-r zap-report.html \
-x zap-report.xml \
-J zap-report.json \
-a \
--hook=/zap/auth_hook.py
artifacts:
reports:
dast: zap-report.json
paths:
- zap-report.html
- zap-report.xml
expire_in: 1 week
allow_failure: true
Container Security Scanning
# Container scanning configuration
include:
- template: Security/Container-Scanning.gitlab-ci.yml
variables:
CS_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
CS_DISABLE_DEPENDENCY_LIST: "false"
CS_SEVERITY_THRESHOLD: "HIGH"
# Custom Trivy scan
trivy_scan:
stage: security
image: aquasec/trivy:latest
services:
- docker:20.10.16-dind
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "/certs"
before_script:
- export TRIVY_USERNAME="$CI_REGISTRY_USER"
- export TRIVY_PASSWORD="$CI_REGISTRY_PASSWORD"
script:
- |
# Scan for vulnerabilities
trivy image \
--format json \
--output trivy-report.json \
--severity HIGH,CRITICAL \
$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
# Scan for secrets
trivy config \
--format json \
--output trivy-config-report.json \
.
# Scan for misconfigurations
trivy fs \
--format json \
--output trivy-fs-report.json \
--security-checks config,secret \
.
artifacts:
reports:
container_scanning: trivy-report.json
paths:
- trivy-*.json
expire_in: 1 week
# Admission controller policy check
opa_policy_check:
stage: security
image: openpolicyagent/opa:latest
script:
- |
# Download Kubernetes manifests
helm template myapp ./chart > k8s-manifests.yaml
# Run OPA policy checks
opa eval -d policies/ -i k8s-manifests.yaml \
"data.kubernetes.admission.deny[x]" > policy-violations.json
# Check for violations
VIOLATIONS=$(cat policy-violations.json | jq length)
if [ "$VIOLATIONS" -gt 0 ]; then
echo "❌ Policy violations found:"
cat policy-violations.json | jq '.'
exit 1
fi
echo "✅ All OPA policy checks passed"
artifacts:
paths:
- policy-violations.json
expire_in: 1 week
Compliance and Governance
FIPS 140-2 Compliance Pipeline
# FIPS compliance validation
fips_compliance_check:
stage: security
image: registry.access.redhat.com/ubi8/ubi:latest
script:
- |
# Verify FIPS mode is enabled
if [ ! -f /proc/sys/crypto/fips_enabled ] || [ "$(cat /proc/sys/crypto/fips_enabled)" != "1" ]; then
echo "❌ FIPS mode not enabled"
exit 1
fi
# Check cryptographic libraries
rpm -qa | grep -E "(openssl|nss|gnutls)" > crypto-packages.txt
# Validate approved algorithms only
./scripts/validate-crypto-algorithms.sh
echo "✅ FIPS 140-2 compliance validated"
artifacts:
paths:
- crypto-packages.txt
expire_in: 30 days
only:
variables:
- $FIPS_REQUIRED == "true"
SOC 2 Compliance Automation
# SOC 2 compliance checks
soc2_compliance:
stage: security
image: alpine:latest
before_script:
- apk add --no-cache curl jq git
script:
- |
# CC6.1 - Logical and physical access controls
echo "Checking access controls..."
./scripts/check-access-controls.sh
# CC6.7 - System operations
echo "Validating system operations..."
./scripts/validate-operations.sh
# CC7.1 - System monitoring
echo "Verifying monitoring capabilities..."
./scripts/check-monitoring.sh
# A1.2 - Availability monitoring
echo "Checking availability metrics..."
./scripts/check-availability.sh
echo "✅ SOC 2 compliance checks completed"
artifacts:
reports:
junit: soc2-compliance-report.xml
paths:
- compliance-evidence/
expire_in: 1 year
Project Management & Collaboration
Issue Tracking and Planning
Epic and Issue Management
# GitLab CLI commands for project management
# Create an epic
glab epic create --title "User Authentication System" \
--description "Implement comprehensive user authentication with SSO support"
# Create issues under the epic
glab issue create --title "Design authentication API endpoints" \
--description "Define REST API endpoints for user authentication" \
--label "backend,api" \
--assignee "backend-dev" \
--milestone "Sprint 1"
glab issue create --title "Implement OAuth2 integration" \
--description "Add OAuth2 support for Google and GitHub" \
--label "backend,oauth" \
--assignee "backend-dev" \
--milestone "Sprint 2"
# Link issues to epic via API
curl --request POST --header "PRIVATE-TOKEN: your-token" \
"https://gitlab.example.com/api/v4/groups/GROUP_ID/epics/EPIC_ID/issues/ISSUE_ID"
Advanced Issue Templates
Create .gitlab/issue_templates/Bug.md
:
## Bug Report
### Summary
Brief description of the bug
### Environment
- GitLab version:
- Browser:
- OS:
### Steps to Reproduce
1.
2.
3.
### Expected Behavior
What you expected to happen
### Actual Behavior
What actually happened
### Screenshots/Logs
Add screenshots or error logs here
### Additional Context
Add any other context about the problem here
### Definition of Done
- [ ] Bug is reproduced
- [ ] Root cause is identified
- [ ] Fix is implemented
- [ ] Tests are added
- [ ] Documentation is updated
- [ ] Code is reviewed and merged
---
/label ~bug ~needs-investigation
/assign @team-lead
Create .gitlab/issue_templates/Feature.md
:
## Feature Request
### User Story
As a [user type], I want [functionality] so that [benefit/value]
### Acceptance Criteria
- [ ] Criteria 1
- [ ] Criteria 2
- [ ] Criteria 3
### Technical Requirements
- [ ] API endpoints defined
- [ ] Database schema updated
- [ ] Frontend components created
- [ ] Tests implemented
- [ ] Documentation updated
### Design Mockups
Add design mockups or wireframes here
### Technical Approach
Brief description of technical implementation
### Dependencies
List any dependencies or blockers
### Effort Estimation
- Story Points:
- Time Estimate:
---
/label ~feature ~needs-design
/milestone %"Sprint X"
Advanced Merge Request Workflows
Merge Request Templates
Create .gitlab/merge_request_templates/Default.md
:
## Merge Request Checklist
### Changes Made
- [ ] What changes were made
- [ ] Why these changes were necessary
- [ ] Impact on existing functionality
### Testing
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Manual testing completed
- [ ] Performance impact assessed
### Code Quality
- [ ] Code follows project standards
- [ ] No new security vulnerabilities introduced
- [ ] Documentation updated
- [ ] Comments added where necessary
### Deployment
- [ ] Database migrations included (if applicable)
- [ ] Environment variables updated (if applicable)
- [ ] Configuration changes documented
- [ ] Rollback plan considered
### Review Requirements
- [ ] Code reviewed by team lead
- [ ] Security review completed (if applicable)
- [ ] Architecture review completed (if applicable)
### Screenshots/Evidence
Add screenshots or evidence of functionality here
---
Closes #ISSUE_NUMBER
/assign @reviewer
Advanced Approval Rules
# Configure push rules via API
curl --request POST --header "PRIVATE-TOKEN: your-token" \
"https://gitlab.example.com/api/v4/projects/PROJECT_ID/push_rule" \
--data "commit_message_regex=^(feat|fix|docs|style|refactor|test|chore)(\(.+\))?: .{1,50}" \
--data "branch_name_regex=^(feature|bugfix|hotfix)\/.+$" \
--data "author_email_regex=@yourcompany\.com$" \
--data "file_name_regex=" \
--data "max_file_size=100"
# Configure approval rules
curl --request POST --header "PRIVATE-TOKEN: your-token" \
"https://gitlab.example.com/api/v4/projects/PROJECT_ID/merge_request_approval_rules" \
--data "name=Security Review" \
--data "approvals_required=1" \
--data "user_ids[]=SECURITY_TEAM_USER_ID" \
--data "rule_type=regular"
Code Review Best Practices
Automated Code Review Integration
# Code review automation pipeline
stages:
- review
- approve
# Automated code analysis
code_analysis:
stage: review
image: alpine:latest
before_script:
- apk add --no-cache git jq curl
script:
- |
# Get merge request changes
MR_CHANGES=$(git diff --name-only origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME..HEAD)
# Calculate complexity metrics
echo "Analyzing code complexity..."
./scripts/calculate-complexity.sh > complexity-report.json
# Check for large files
echo "Checking for large files..."
find . -name "*.py" -size +500c > large-files.txt
# Generate review report
cat > review-report.md << EOF
# Automated Code Review Report
## Changes Summary
- Files changed: $(echo "$MR_CHANGES" | wc -l)
- Lines added: $(git diff --numstat origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME..HEAD | awk '{sum+=$1} END {print sum}')
- Lines removed: $(git diff --numstat origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME..HEAD | awk '{sum+=$2} END {print sum}')
## Complexity Analysis
$(cat complexity-report.json | jq -r '.summary')
## Recommendations
$(./scripts/generate-recommendations.sh)
EOF
# Post comment to merge request
curl --request POST --header "PRIVATE-TOKEN: $CI_JOB_TOKEN" \
--header "Content-Type: application/json" \
--data "{\"body\":\"$(cat review-report.md | sed 's/"/\\"/g' | tr '\n' ' ')\"}" \
"$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes"
artifacts:
paths:
- review-report.md
- complexity-report.json
expire_in: 7 days
only:
- merge_requests
GitLab Performance & Scaling
Performance Monitoring
GitLab Metrics and Observability
# Prometheus configuration for GitLab monitoring
global:
scrape_interval: 15s
evaluation_interval: 15s
rule_files:
- "gitlab_rules.yml"
scrape_configs:
- job_name: "gitlab"
static_configs:
- targets: ["gitlab.example.com:9090"]
metrics_path: "/-/metrics"
- job_name: "gitlab-runner"
static_configs:
- targets: ["runner1.example.com:9252", "runner2.example.com:9252"]
- job_name: "postgres"
static_configs:
- targets: ["postgres.example.com:9187"]
- job_name: "redis"
static_configs:
- targets: ["redis.example.com:9121"]
GitLab alerting rules (gitlab_rules.yml
):
groups:
- name: gitlab
rules:
- alert: GitLabDown
expr: up{job="gitlab"} == 0
for: 5m
labels:
severity: critical
annotations:
summary: "GitLab is down"
description: "GitLab has been down for more than 5 minutes"
- alert: GitLabHighMemoryUsage
expr: (gitlab_memory_usage_bytes / gitlab_memory_limit_bytes) > 0.85
for: 10m
labels:
severity: warning
annotations:
summary: "GitLab memory usage is high"
description: "GitLab memory usage is {{ $value | humanizePercentage }}"
- alert: GitLabHighDatabaseConnections
expr: gitlab_database_connections > 80
for: 5m
labels:
severity: warning
annotations:
summary: "GitLab database connections are high"
description: "GitLab has {{ $value }} database connections"
- alert: GitLabRunnerOffline
expr: up{job="gitlab-runner"} == 0
for: 3m
labels:
severity: warning
annotations:
summary: "GitLab Runner is offline"
description: "GitLab Runner {{ $labels.instance }} has been offline for more than 3 minutes"
- alert: GitLabJobQueueHigh
expr: gitlab_runner_jobs{state="running"} > 50
for: 5m
labels:
severity: warning
annotations:
summary: "GitLab job queue is high"
description: "There are {{ $value }} jobs running, which is above the threshold"
Performance Tuning Configuration
# /etc/gitlab/gitlab.rb - Performance optimizations
# Memory allocation
puma['worker_processes'] = 4
puma['min_threads'] = 4
puma['max_threads'] = 4
# Database connection pool
gitlab_rails['db_pool'] = 20
postgresql['shared_buffers'] = "512MB"
postgresql['effective_cache_size'] = "2GB"
postgresql['work_mem'] = "16MB"
postgresql['maintenance_work_mem'] = "128MB"
# Redis configuration
redis['maxmemory'] = "2gb"
redis['maxmemory_policy'] = "allkeys-lru"
# Gitaly (Git storage)
gitaly['ruby_max_rss'] = 300000000 # 300MB
gitaly['ruby_graceful_restart_timeout'] = '10m'
gitaly['ruby_restart_delay'] = '5m'
# Sidekiq (background jobs)
sidekiq['max_concurrency'] = 25
# Nginx optimizations
nginx['worker_processes'] = 4
nginx['worker_connections'] = 1024
nginx['keepalive_timeout'] = 65
nginx['gzip'] = "on"
nginx['gzip_comp_level'] = 6
# Monitoring
prometheus_monitoring['enable'] = true
node_exporter['enable'] = true
postgres_exporter['enable'] = true
redis_exporter['enable'] = true
Scaling Strategies
Horizontal Scaling Architecture
graph TB
LB[Load Balancer] --> |HTTPS| WEB1[GitLab Web 1]
LB --> |HTTPS| WEB2[GitLab Web 2]
LB --> |HTTPS| WEB3[GitLab Web 3]
WEB1 --> |TCP| GITALY1[Gitaly 1]
WEB2 --> |TCP| GITALY2[Gitaly 2]
WEB3 --> |TCP| GITALY3[Gitaly 3]
WEB1 --> |TCP| PG[(PostgreSQL Primary)]
WEB2 --> |TCP| PG
WEB3 --> |TCP| PG
PG --> |Replication| PG_STANDBY[(PostgreSQL Standby)]
WEB1 --> |TCP| REDIS[(Redis Cluster)]
WEB2 --> |TCP| REDIS
WEB3 --> |TCP| REDIS
WEB1 --> |TCP| ES[(Elasticsearch)]
WEB2 --> |TCP| ES
WEB3 --> |TCP| ES
SIDEKIQ1[Sidekiq 1] --> |TCP| REDIS
SIDEKIQ2[Sidekiq 2] --> |TCP| REDIS
SIDEKIQ3[Sidekiq 3] --> |TCP| REDIS
Kubernetes Deployment for Scale
# GitLab Helm values for production deployment
global:
hosts:
domain: gitlab.example.com
edition: ce
# High availability PostgreSQL
psql:
host: postgres.example.com
port: 5432
database: gitlab
username: gitlab
password:
secret: gitlab-postgres-secret
key: password
# Redis cluster
redis:
host: redis-cluster.example.com
port: 6379
auth:
enabled: true
secret: gitlab-redis-secret
key: password
# Object storage
minio:
enabled: false
appConfig:
object_store:
enabled: true
connection:
secret: gitlab-objectstore-secret
artifacts:
bucket: gitlab-artifacts
uploads:
bucket: gitlab-uploads
packages:
bucket: gitlab-packages
backups:
bucket: gitlab-backups
# GitLab application scaling
gitlab:
webservice:
replicaCount: 3
resources:
requests:
cpu: 1
memory: 2Gi
limits:
cpu: 2
memory: 4Gi
sidekiq:
replicaCount: 3
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 1
memory: 2Gi
# Gitaly cluster configuration
gitlab:
gitaly:
enabled: false
gitaly:
enabled: true
replicaCount: 3
persistence:
enabled: true
size: 500Gi
storageClass: ssd
resources:
requests:
cpu: 1
memory: 2Gi
limits:
cpu: 2
memory: 4Gi
# Registry scaling
registry:
enabled: true
replicaCount: 2
storage:
secret: gitlab-registry-storage
resources:
requests:
cpu: 200m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
# Runner auto-scaling
gitlab-runner:
replicas: 3
concurrent: 30
checkInterval: 3
runners:
config: |
[[runners]]
[runners.kubernetes]
image = "ubuntu:20.04"
privileged = true
# Auto-scaling configuration
request_concurrency = 10
# Resource limits
cpu_limit = "1"
memory_limit = "2Gi"
service_cpu_limit = "0.5"
service_memory_limit = "512Mi"
Database Scaling and Optimization
-- PostgreSQL performance tuning queries
-- Find slow queries
SELECT
query,
calls,
total_time,
mean_time,
rows
FROM pg_stat_statements
ORDER BY total_time DESC
LIMIT 10;
-- Index usage analysis
SELECT
schemaname,
tablename,
attname,
n_distinct,
correlation
FROM pg_stats
WHERE schemaname = 'public';
-- Table sizes and bloat analysis
SELECT
schemaname,
tablename,
pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) as size
FROM pg_tables
WHERE schemaname = 'public'
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC;
PostgreSQL configuration for GitLab:
# postgresql.conf optimizations for GitLab
shared_buffers = 8GB # 25% of total RAM
effective_cache_size = 24GB # 75% of total RAM
work_mem = 64MB # For sorting operations
maintenance_work_mem = 2GB # For VACUUM, CREATE INDEX
random_page_cost = 1.1 # For SSD storage
effective_io_concurrency = 200 # For SSD storage
# Connection settings
max_connections = 500
shared_preload_libraries = 'pg_stat_statements'
# WAL settings
wal_buffers = 64MB
checkpoint_completion_target = 0.9
max_wal_size = 4GB
min_wal_size = 512MB
# Query planner
default_statistics_target = 1000
Migration Strategies
GitHub to GitLab Migration
Complete Migration Script
#!/bin/bash
# github-to-gitlab-migration.sh
# Complete migration script from GitHub to GitLab
set -e
# Configuration
GITHUB_ORG="your-github-org"
GITLAB_GROUP="your-gitlab-group"
GITHUB_TOKEN="your-github-token"
GITLAB_TOKEN="your-gitlab-token"
GITLAB_URL="https://gitlab.example.com"
echo "🚀 Starting GitHub to GitLab migration"
# Get list of repositories from GitHub
echo "📋 Fetching repositories from GitHub..."
REPOS=$(curl -s -H "Authorization: token $GITHUB_TOKEN" \
"https://api.github.com/orgs/$GITHUB_ORG/repos?per_page=100&type=all" | \
jq -r '.[].name')
for repo in $REPOS; do
echo "🔄 Processing repository: $repo"
# Create GitLab project
echo " 📁 Creating GitLab project..."
PROJECT_DATA=$(cat <<EOF
{
"name": "$repo",
"namespace_id": "$GITLAB_GROUP_ID",
"description": "Migrated from GitHub",
"visibility": "private",
"issues_enabled": true,
"merge_requests_enabled": true,
"wiki_enabled": true,
"snippets_enabled": true
}
EOF
)
GITLAB_PROJECT_ID=$(curl -s -X POST \
-H "Authorization: Bearer $GITLAB_TOKEN" \
-H "Content-Type: application/json" \
-d "$PROJECT_DATA" \
"$GITLAB_URL/api/v4/projects" | jq -r '.id')
# Clone from GitHub
echo " ⬇️ Cloning from GitHub..."
git clone --mirror "https://$GITHUB_TOKEN@github.com/$GITHUB_ORG/$repo.git" "$repo.git"
cd "$repo.git"
# Push to GitLab
echo " ⬆️ Pushing to GitLab..."
git remote set-url origin "$GITLAB_URL/$GITLAB_GROUP/$repo.git"
git push --mirror origin
cd ..
rm -rf "$repo.git"
# Migrate issues using GitLab import feature
echo " 🐛 Importing issues..."
curl -X POST \
-H "Authorization: Bearer $GITLAB_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"personal_access_token\": \"$GITHUB_TOKEN\"}" \
"$GITLAB_URL/api/v4/projects/$GITLAB_PROJECT_ID/import/github"
echo " ✅ Repository $repo migrated successfully"
sleep 2 # Rate limiting
done
echo "🎉 Migration completed!"
Migration Verification Script
#!/usr/bin/env python3
"""
GitLab migration verification script
Compares GitHub and GitLab repositories to ensure complete migration
"""
import requests
import json
import sys
from typing import Dict, List
class MigrationVerifier:
def __init__(self, github_token: str, gitlab_token: str, gitlab_url: str):
self.github_token = github_token
self.gitlab_token = gitlab_token
self.gitlab_url = gitlab_url
self.github_headers = {"Authorization": f"token {github_token}"}
self.gitlab_headers = {"Authorization": f"Bearer {gitlab_token}"}
def get_github_repos(self, org: str) -> List[Dict]:
"""Get all repositories from GitHub organization."""
url = f"https://api.github.com/orgs/{org}/repos"
response = requests.get(url, headers=self.github_headers)
response.raise_for_status()
return response.json()
def get_gitlab_projects(self, group_id: int) -> List[Dict]:
"""Get all projects from GitLab group."""
url = f"{self.gitlab_url}/api/v4/groups/{group_id}/projects"
response = requests.get(url, headers=self.gitlab_headers)
response.raise_for_status()
return response.json()
def compare_repository(self, github_repo: str, gitlab_project_id: int) -> Dict:
"""Compare a GitHub repository with GitLab project."""
results = {
"repository": github_repo,
"status": "success",
"issues": []
}
# Get GitHub repository details
gh_url = f"https://api.github.com/repos/{github_repo}"
gh_response = requests.get(gh_url, headers=self.github_headers)
gh_data = gh_response.json()
# Get GitLab project details
gl_url = f"{self.gitlab_url}/api/v4/projects/{gitlab_project_id}"
gl_response = requests.get(gl_url, headers=self.gitlab_headers)
gl_data = gl_response.json()
# Compare branches
gh_branches = self._get_github_branches(github_repo)
gl_branches = self._get_gitlab_branches(gitlab_project_id)
if len(gh_branches) != len(gl_branches):
results["issues"].append(
f"Branch count mismatch: GitHub {len(gh_branches)}, GitLab {len(gl_branches)}"
)
# Compare tags
gh_tags = self._get_github_tags(github_repo)
gl_tags = self._get_gitlab_tags(gitlab_project_id)
if len(gh_tags) != len(gl_tags):
results["issues"].append(
f"Tag count mismatch: GitHub {len(gh_tags)}, GitLab {len(gl_tags)}"
)
# Compare issues
gh_issues = self._get_github_issues(github_repo)
gl_issues = self._get_gitlab_issues(gitlab_project_id)
if len(gh_issues) != len(gl_issues):
results["issues"].append(
f"Issue count mismatch: GitHub {len(gh_issues)}, GitLab {len(gl_issues)}"
)
if results["issues"]:
results["status"] = "warning"
return results
def _get_github_branches(self, repo: str) -> List[Dict]:
"""Get branches from GitHub repository."""
url = f"https://api.github.com/repos/{repo}/branches"
response = requests.get(url, headers=self.github_headers)
return response.json() if response.status_code == 200 else []
def _get_gitlab_branches(self, project_id: int) -> List[Dict]:
"""Get branches from GitLab project."""
url = f"{self.gitlab_url}/api/v4/projects/{project_id}/repository/branches"
response = requests.get(url, headers=self.gitlab_headers)
return response.json() if response.status_code == 200 else []
def _get_github_tags(self, repo: str) -> List[Dict]:
"""Get tags from GitHub repository."""
url = f"https://api.github.com/repos/{repo}/tags"
response = requests.get(url, headers=self.github_headers)
return response.json() if response.status_code == 200 else []
def _get_gitlab_tags(self, project_id: int) -> List[Dict]:
"""Get tags from GitLab project."""
url = f"{self.gitlab_url}/api/v4/projects/{project_id}/repository/tags"
response = requests.get(url, headers=self.gitlab_headers)
return response.json() if response.status_code == 200 else []
def _get_github_issues(self, repo: str) -> List[Dict]:
"""Get issues from GitHub repository."""
url = f"https://api.github.com/repos/{repo}/issues"
response = requests.get(url, headers=self.github_headers,
params={"state": "all", "per_page": 100})
return response.json() if response.status_code == 200 else []
def _get_gitlab_issues(self, project_id: int) -> List[Dict]:
"""Get issues from GitLab project."""
url = f"{self.gitlab_url}/api/v4/projects/{project_id}/issues"
response = requests.get(url, headers=self.gitlab_headers,
params={"state": "all", "per_page": 100})
return response.json() if response.status_code == 200 else []
def main():
verifier = MigrationVerifier(
github_token="your-github-token",
gitlab_token="your-gitlab-token",
gitlab_url="https://gitlab.example.com"
)
# Define repositories to verify
repositories = [
{"github": "your-org/repo1", "gitlab_project_id": 123},
{"github": "your-org/repo2", "gitlab_project_id": 124},
]
print("🔍 Starting migration verification...")
all_results = []
for repo_config in repositories:
result = verifier.compare_repository(
repo_config["github"],
repo_config["gitlab_project_id"]
)
all_results.append(result)
status_emoji = "✅" if result["status"] == "success" else "⚠️"
print(f"{status_emoji} {result['repository']}: {result['status']}")
for issue in result["issues"]:
print(f" - {issue}")
# Generate summary report
success_count = sum(1 for r in all_results if r["status"] == "success")
warning_count = len(all_results) - success_count
print(f"\n📊 Migration Verification Summary:")
print(f" ✅ Successful: {success_count}")
print(f" ⚠️ With warnings: {warning_count}")
print(f" 📊 Total verified: {len(all_results)}")
# Exit with appropriate code
sys.exit(0 if warning_count == 0 else 1)
if __name__ == "__main__":
main()
Legacy System Integration
Subversion to GitLab Migration
#!/bin/bash
# svn-to-gitlab-migration.sh
set -e
SVN_REPO_URL="https://svn.example.com/repo"
GITLAB_PROJECT_NAME="migrated-project"
GITLAB_GROUP="legacy-projects"
echo "🔄 Starting SVN to GitLab migration"
# Create authors file for SVN
echo "📝 Creating authors mapping..."
cat > authors.txt << EOF
svn-user1 = John Doe <[email protected]>
svn-user2 = Jane Smith <[email protected]>
(no author) = Legacy User <[email protected]>
EOF
# Clone SVN repository with full history
echo "⬇️ Cloning SVN repository..."
git svn clone "$SVN_REPO_URL" \
--authors-file=authors.txt \
--no-metadata \
--trunk=trunk \
--branches=branches \
--tags=tags \
"$GITLAB_PROJECT_NAME"
cd "$GITLAB_PROJECT_NAME"
# Convert SVN tags to Git tags
echo "🏷️ Converting SVN tags to Git tags..."
for tag in $(git branch -r | grep "tags/" | sed 's/.*tags\///'); do
git tag "$tag" "refs/remotes/tags/$tag"
git branch -D -r "tags/$tag"
done
# Convert SVN branches to Git branches
echo "🌿 Converting SVN branches..."
for branch in $(git branch -r | grep -v tags/ | grep -v trunk | sed 's/.*origin\///'); do
git checkout -b "$branch" "refs/remotes/$branch"
done
# Checkout main branch
git checkout master
git branch -m master main
# Clean up remote references
git remote rm origin 2>/dev/null || true
# Add GitLab remote and push
echo "⬆️ Pushing to GitLab..."
git remote add origin "$GITLAB_URL/$GITLAB_GROUP/$GITLAB_PROJECT_NAME.git"
git push -u origin --all
git push origin --tags
echo "✅ SVN migration completed successfully"
Incremental Migration Strategies
Phased Migration Approach
# .gitlab-ci.yml for phased migration
stages:
- validate
- migrate
- verify
- cleanup
variables:
MIGRATION_PHASE: "1" # Can be 1, 2, or 3
# Phase 1: Critical repositories
migrate_phase_1:
stage: migrate
script:
- |
if [ "$MIGRATION_PHASE" == "1" ]; then
echo "🚀 Starting Phase 1: Critical repositories"
./scripts/migrate-critical-repos.sh
fi
only:
variables:
- $MIGRATION_PHASE == "1"
# Phase 2: Development repositories
migrate_phase_2:
stage: migrate
script:
- |
if [ "$MIGRATION_PHASE" == "2" ]; then
echo "🚀 Starting Phase 2: Development repositories"
./scripts/migrate-dev-repos.sh
fi
only:
variables:
- $MIGRATION_PHASE == "2"
# Phase 3: Archive repositories
migrate_phase_3:
stage: migrate
script:
- |
if [ "$MIGRATION_PHASE" == "3" ]; then
echo "🚀 Starting Phase 3: Archive repositories"
./scripts/migrate-archive-repos.sh
fi
only:
variables:
- $MIGRATION_PHASE == "3"
# Validation after each phase
validate_migration:
stage: verify
script:
- ./scripts/validate-migration.py --phase $MIGRATION_PHASE
artifacts:
reports:
junit: migration-validation-report.xml
paths:
- migration-report.html
Best Practices & Real-World Examples
Enterprise Implementation Case Study
Fortune 500 Company Migration
Company Profile:
- 5,000+ developers across 50 teams
- 2,000+ repositories
- Multiple compliance requirements (SOX, HIPAA, PCI-DSS)
- Global development across 12 time zones
Implementation Strategy:
# Enterprise GitLab configuration structure
# .gitlab/
# ├── ci/
# │ ├── templates/
# │ │ ├── security.yml
# │ │ ├── compliance.yml
# │ │ └── deployment.yml
# │ └── rules/
# │ ├── branch-protection.yml
# │ └── approval-rules.yml
# ├── templates/
# │ ├── issues/
# │ └── merge_requests/
# └── policies/
# ├── security-policy.yml
# └── compliance-policy.yml
Security Template (.gitlab/ci/templates/security.yml
):
# Enterprise security pipeline template
.security_template:
rules:
- if: "$CI_MERGE_REQUEST_IID"
- if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
# Multi-stage security scanning
include:
- template: Security/SAST.gitlab-ci.yml
- template: Security/DAST.gitlab-ci.yml
- template: Security/Dependency-Scanning.gitlab-ci.yml
- template: Security/Container-Scanning.gitlab-ci.yml
- template: Security/Secret-Detection.gitlab-ci.yml
# Custom enterprise security job
enterprise_security_scan:
extends: .security_template
stage: security
image: enterprise-security-scanner:latest
script:
- |
# Custom security tools integration
enterprise-scanner --format json --output security-report.json .
# HIPAA compliance check
hipaa-validator --config .hipaa-config.yml --output hipaa-report.xml
# PCI-DSS scanning
pci-scanner --level 1 --output pci-report.json
# Upload results to enterprise security dashboard
curl -X POST -H "Authorization: Bearer $SECURITY_DASHBOARD_TOKEN" \
-F "[email protected]" \
-F "project_id=$CI_PROJECT_ID" \
"$ENTERPRISE_SECURITY_DASHBOARD_URL/api/v1/reports"
artifacts:
reports:
sast: security-report.json
junit: hipaa-report.xml
expire_in: 30 days
allow_failure: false
# Compliance validation
compliance_check:
extends: .security_template
stage: security
script:
- |
# SOX compliance validation
sox-validator --config .sox-config.yml
# Data classification check
data-classifier --scan-path . --output classification-report.json
# Audit trail generation
audit-generator --project-id $CI_PROJECT_ID --commit-sha $CI_COMMIT_SHA
artifacts:
paths:
- classification-report.json
- audit-trail.log
expire_in: 7 years # SOX requirement
Multi-Team Workflow Implementation
# Team-specific workflow configuration
# Marketing team workflow
.marketing_workflow:
rules:
- if: '$CI_COMMIT_BRANCH =~ /^marketing\/.*/'
marketing_review:
extends: .marketing_workflow
stage: review
script:
- |
# Content review with marketing tools
content-validator --type marketing --path content/
brand-compliance-checker --config .brand-config.yml
only:
- merge_requests
# Engineering team workflow
.engineering_workflow:
rules:
- if: '$CI_COMMIT_BRANCH =~ /^(feature|bugfix|hotfix)\/.*/'
code_quality:
extends: .engineering_workflow
stage: test
script:
- |
# Engineering-specific quality gates
code-complexity-analyzer --threshold 10
architecture-validator --config .architecture-rules.yml
performance-regression-test --baseline production
# DevOps team workflow
.devops_workflow:
rules:
- if: '$CI_COMMIT_BRANCH =~ /^(infrastructure|deployment)\/.*/'
infrastructure_validation:
extends: .devops_workflow
stage: test
script:
- |
# Infrastructure as Code validation
terraform validate
terraform plan -out=tfplan
checkov -f tfplan --framework terraform
cost-estimator --plan tfplan --output cost-report.json
Development Team Workflow Examples
Agile Sprint Integration
#!/usr/bin/env python3
"""
GitLab Agile Sprint Integration
Automatically manages sprint workflows and reporting
"""
import requests
import json
from datetime import datetime, timedelta
from typing import List, Dict
class GitLabSprintManager:
def __init__(self, gitlab_url: str, token: str, project_id: int):
self.gitlab_url = gitlab_url
self.token = token
self.project_id = project_id
self.headers = {"Authorization": f"Bearer {token}"}
def create_sprint_milestone(self, sprint_name: str, duration_weeks: int = 2) -> int:
"""Create a new sprint milestone."""
start_date = datetime.now().date()
end_date = start_date + timedelta(weeks=duration_weeks)
milestone_data = {
"title": sprint_name,
"description": f"Sprint from {start_date} to {end_date}",
"due_date": end_date.isoformat(),
"start_date": start_date.isoformat()
}
response = requests.post(
f"{self.gitlab_url}/api/v4/projects/{self.project_id}/milestones",
headers=self.headers,
json=milestone_data
)
return response.json()["id"]
def assign_issues_to_sprint(self, milestone_id: int, issue_ids: List[int]):
"""Assign issues to sprint milestone."""
for issue_id in issue_ids:
issue_data = {"milestone_id": milestone_id}
requests.put(
f"{self.gitlab_url}/api/v4/projects/{self.project_id}/issues/{issue_id}",
headers=self.headers,
json=issue_data
)
def generate_sprint_report(self, milestone_id: int) -> Dict:
"""Generate comprehensive sprint report."""
# Get milestone details
milestone = requests.get(
f"{self.gitlab_url}/api/v4/projects/{self.project_id}/milestones/{milestone_id}",
headers=self.headers
).json()
# Get issues in milestone
issues = requests.get(
f"{self.gitlab_url}/api/v4/projects/{self.project_id}/issues",
headers=self.headers,
params={"milestone": milestone["title"], "per_page": 100}
).json()
# Calculate sprint metrics
total_issues = len(issues)
closed_issues = len([i for i in issues if i["state"] == "closed"])
open_issues = total_issues - closed_issues
# Calculate story points (from issue labels)
total_points = 0
completed_points = 0
for issue in issues:
points = self._extract_story_points(issue["labels"])
total_points += points
if issue["state"] == "closed":
completed_points += points
return {
"milestone": milestone["title"],
"start_date": milestone.get("start_date"),
"due_date": milestone.get("due_date"),
"total_issues": total_issues,
"closed_issues": closed_issues,
"open_issues": open_issues,
"completion_rate": (closed_issues / total_issues * 100) if total_issues > 0 else 0,
"total_story_points": total_points,
"completed_story_points": completed_points,
"velocity": completed_points,
"issues_by_assignee": self._group_issues_by_assignee(issues),
"burndown_data": self._calculate_burndown(milestone_id)
}
def _extract_story_points(self, labels: List[str]) -> int:
"""Extract story points from issue labels."""
for label in labels:
if label.startswith("SP:"):
try:
return int(label.split(":")[1])
except ValueError:
continue
return 0
def _group_issues_by_assignee(self, issues: List[Dict]) -> Dict:
"""Group issues by assignee for team analysis."""
assignee_data = {}
for issue in issues:
assignee = issue.get("assignee")
if assignee:
name = assignee["name"]
if name not in assignee_data:
assignee_data[name] = {"total": 0, "closed": 0, "points": 0}
assignee_data[name]["total"] += 1
if issue["state"] == "closed":
assignee_data[name]["closed"] += 1
assignee_data[name]["points"] += self._extract_story_points(issue["labels"])
return assignee_data
def _calculate_burndown(self, milestone_id: int) -> List[Dict]:
"""Calculate burndown chart data."""
# This would typically query issue state changes over time
# For simplicity, showing the structure
return [
{"date": "2025-08-01", "remaining_points": 50},
{"date": "2025-08-02", "remaining_points": 45},
{"date": "2025-08-03", "remaining_points": 40},
# ... continue for sprint duration
]
# Example usage
def main():
sprint_manager = GitLabSprintManager(
gitlab_url="https://gitlab.example.com",
token="your-token",
project_id=123
)
# Create new sprint
milestone_id = sprint_manager.create_sprint_milestone("Sprint 2025-32", duration_weeks=2)
# Assign issues to sprint
sprint_manager.assign_issues_to_sprint(milestone_id, [101, 102, 103, 104])
# Generate report at sprint end
report = sprint_manager.generate_sprint_report(milestone_id)
print(f"📊 Sprint Report: {report['milestone']}")
print(f" Completion Rate: {report['completion_rate']:.1f}%")
print(f" Velocity: {report['velocity']} story points")
print(f" Issues: {report['closed_issues']}/{report['total_issues']} completed")
if __name__ == "__main__":
main()
GitLab Flow Implementation
# .gitlab-ci.yml implementing GitLab Flow
stages:
- validate
- test
- build
- security
- staging
- production
variables:
DOCKER_DRIVER: overlay2
# Branch-specific rules
.feature_branch_rules: &feature_rules
rules:
- if: "$CI_MERGE_REQUEST_IID"
- if: '$CI_COMMIT_BRANCH =~ /^feature\/.*/'
.main_branch_rules: &main_rules
rules:
- if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
.production_rules: &production_rules
rules:
- if: '$CI_COMMIT_BRANCH == "production"'
# Feature branch pipeline
feature_validation:
<<: *feature_rules
stage: validate
script:
- echo "Validating feature branch: $CI_COMMIT_REF_NAME"
- ./scripts/validate-branch-name.sh
- ./scripts/validate-commit-message.sh
feature_test:
<<: *feature_rules
stage: test
script:
- npm install
- npm run test:unit
- npm run test:integration
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml
# Main branch pipeline
main_build:
<<: *main_rules
stage: build
script:
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
- docker push $CI_REGISTRY_IMAGE:latest
deploy_staging:
<<: *main_rules
stage: staging
environment:
name: staging
url: https://staging.example.com
deployment_tier: staging
script:
- kubectl set image deployment/app app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- kubectl rollout status deployment/app
- ./scripts/smoke-test.sh staging.example.com
# Production deployment (from production branch)
deploy_production:
<<: *production_rules
stage: production
environment:
name: production
url: https://production.example.com
deployment_tier: production
script:
- kubectl set image deployment/app app=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
- kubectl rollout status deployment/app
- ./scripts/smoke-test.sh production.example.com
when: manual
# Automatic merge to production branch
create_production_mr:
<<: *main_rules
stage: production
image: alpine/git:latest
before_script:
- apk add --no-cache curl jq
script:
- |
# Create MR from main to production
MR_DATA=$(cat <<EOF
{
"source_branch": "$CI_DEFAULT_BRANCH",
"target_branch": "production",
"title": "Release: Deploy $CI_COMMIT_SHA to production",
"description": "Automated release MR for commit $CI_COMMIT_SHA"
}
EOF
)
curl -X POST \
-H "Authorization: Bearer $CI_JOB_TOKEN" \
-H "Content-Type: application/json" \
-d "$MR_DATA" \
"$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests"
only:
- main
Code Quality and Standards
Comprehensive Code Quality Pipeline
# .gitlab-ci.yml for code quality
stages:
- lint
- test
- quality
- security
# Language-specific linting jobs
eslint:
stage: lint
image: node:16-alpine
script:
- npm ci
- npm run lint:js
- npm run lint:css
artifacts:
reports:
codequality: eslint-report.json
rules:
- changes:
- "**/*.js"
- "**/*.ts"
- "**/*.vue"
- "**/*.css"
- "**/*.scss"
flake8:
stage: lint
image: python:3.9-alpine
before_script:
- pip install flake8 flake8-json
script:
- flake8 --format=json --output-file=flake8-report.json src/
artifacts:
reports:
codequality: flake8-report.json
rules:
- changes:
- "**/*.py"
golangci_lint:
stage: lint
image: golangci/golangci-lint:latest
script:
- golangci-lint run --out-format json > golangci-report.json
artifacts:
reports:
codequality: golangci-report.json
rules:
- changes:
- "**/*.go"
# Code complexity analysis
complexity_analysis:
stage: quality
image: alpine:latest
before_script:
- apk add --no-cache python3 py3-pip
- pip3 install radon
script:
- |
# Calculate cyclomatic complexity
radon cc src/ --json > complexity-report.json
# Calculate maintainability index
radon mi src/ --json > maintainability-report.json
# Generate HTML report
cat > complexity-summary.html << EOF
<html><body>
<h1>Code Complexity Report</h1>
<h2>Cyclomatic Complexity</h2>
<pre>$(cat complexity-report.json | jq '.')</pre>
<h2>Maintainability Index</h2>
<pre>$(cat maintainability-report.json | jq '.')</pre>
</body></html>
EOF
artifacts:
paths:
- complexity-report.json
- maintainability-report.json
- complexity-summary.html
expire_in: 30 days
# Test coverage analysis
coverage_analysis:
stage: quality
image: python:3.9
script:
- pip install pytest pytest-cov coverage
- pytest --cov=src --cov-report=xml --cov-report=html --cov-report=term
- coverage json --pretty-print
coverage: "/TOTAL.+ ([0-9]{1,3}%)/"
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage.xml
paths:
- htmlcov/
- coverage.json
expire_in: 30 days
# Dependency vulnerability scanning
dependency_scan:
stage: security
image: python:3.9-alpine
before_script:
- pip install safety pip-audit
script:
- |
# Scan Python dependencies
safety check --json --output safety-report.json || true
pip-audit --format=json --output=pip-audit-report.json || true
# Generate combined report
python3 << EOF
import json
# Load reports
try:
with open('safety-report.json') as f:
safety_data = json.load(f)
except:
safety_data = []
try:
with open('pip-audit-report.json') as f:
audit_data = json.load(f)
except:
audit_data = []
# Combine and generate summary
total_vulnerabilities = len(safety_data) + len(audit_data.get('vulnerabilities', []))
with open('security-summary.json', 'w') as f:
json.dump({
'total_vulnerabilities': total_vulnerabilities,
'safety_issues': len(safety_data),
'audit_issues': len(audit_data.get('vulnerabilities', [])),
'reports': {
'safety': safety_data,
'audit': audit_data
}
}, f, indent=2)
print(f"Found {total_vulnerabilities} total vulnerabilities")
EOF
artifacts:
paths:
- safety-report.json
- pip-audit-report.json
- security-summary.json
expire_in: 7 days
# Performance benchmarking
performance_test:
stage: quality
image: node:16
services:
- redis:6-alpine
- postgres:13-alpine
variables:
POSTGRES_DB: testdb
POSTGRES_USER: testuser
POSTGRES_PASSWORD: testpass
script:
- npm ci
- npm run build
- |
# Load testing with Artillery
npx artillery quick --count 50 --num 10 http://localhost:3000 > load-test-results.json
# Memory leak detection
node --expose-gc --inspect ./scripts/memory-leak-test.js > memory-report.json
# Bundle size analysis
npx webpack-bundle-analyzer build/static/js/*.js --report > bundle-analysis.html
artifacts:
paths:
- load-test-results.json
- memory-report.json
- bundle-analysis.html
expire_in: 7 days
Automated Code Review Bot Integration
#!/usr/bin/env python3
"""
Automated Code Review Bot for GitLab
Provides intelligent feedback on merge requests
"""
import requests
import json
import re
from typing import Dict, List
import subprocess
class GitLabCodeReviewBot:
def __init__(self, gitlab_url: str, token: str, project_id: int):
self.gitlab_url = gitlab_url
self.token = token
self.project_id = project_id
self.headers = {"Authorization": f"Bearer {token}"}
def analyze_merge_request(self, mr_iid: int) -> Dict:
"""Analyze merge request and provide feedback."""
# Get MR details
mr_url = f"{self.gitlab_url}/api/v4/projects/{self.project_id}/merge_requests/{mr_iid}"
mr_data = requests.get(mr_url, headers=self.headers).json()
# Get MR changes
changes_url = f"{mr_url}/changes"
changes_data = requests.get(changes_url, headers=self.headers).json()
analysis = {
"mr_title": mr_data["title"],
"source_branch": mr_data["source_branch"],
"target_branch": mr_data["target_branch"],
"changes_count": len(changes_data["changes"]),
"additions": mr_data["changes_count"].split("+")[0],
"deletions": mr_data["changes_count"].split("-")[0],
"issues": []
}
# Analyze changes
for change in changes_data["changes"]:
file_path = change["new_path"]
# Check file size
if change.get("diff"):
lines_changed = len(change["diff"].split("\n"))
if lines_changed > 500:
analysis["issues"].append({
"type": "large_file_change",
"file": file_path,
"message": f"Large change detected ({lines_changed} lines). Consider breaking into smaller commits."
})
# Check for sensitive patterns
if change.get("diff"):
sensitive_patterns = [
(r'password\s*=\s*[\'"][^\'"]*[\'"]', "Hardcoded password detected"),
(r'api_key\s*=\s*[\'"][^\'"]*[\'"]', "Hardcoded API key detected"),
(r'console\.log\(', "Console.log statement found"),
(r'TODO|FIXME|HACK', "TODO/FIXME/HACK comment found"),
(r'\.only\(|\.skip\(', "Test isolation (.only/.skip) detected")
]
for pattern, message in sensitive_patterns:
if re.search(pattern, change["diff"], re.IGNORECASE):
analysis["issues"].append({
"type": "code_quality",
"file": file_path,
"message": message
})
return analysis
def post_review_comment(self, mr_iid: int, analysis: Dict):
"""Post automated review comment."""
if not analysis["issues"]:
comment_body = "✅ **Automated Review: PASSED**\n\nNo issues detected in this merge request."
else:
comment_body = "⚠️ **Automated Review: Issues Found**\n\n"
for issue in analysis["issues"]:
comment_body += f"- **{issue['file']}**: {issue['message']}\n"
comment_body += "\nPlease review these items before merging."
# Add statistics
comment_body += f"\n\n📊 **Change Statistics:**\n"
comment_body += f"- Files changed: {analysis['changes_count']}\n"
comment_body += f"- Lines added: {analysis['additions']}\n"
comment_body += f"- Lines deleted: {analysis['deletions']}\n"
# Post comment
comment_url = f"{self.gitlab_url}/api/v4/projects/{self.project_id}/merge_requests/{mr_iid}/notes"
comment_data = {"body": comment_body}
response = requests.post(comment_url, headers=self.headers, json=comment_data)
return response.status_code == 201
def check_branch_naming(self, branch_name: str) -> List[str]:
"""Check if branch follows naming conventions."""
issues = []
valid_prefixes = ["feature/", "bugfix/", "hotfix/", "release/", "chore/"]
if not any(branch_name.startswith(prefix) for prefix in valid_prefixes):
issues.append("Branch name should start with: feature/, bugfix/, hotfix/, release/, or chore/")
if len(branch_name.split("/")) < 2:
issues.append("Branch name should include descriptive suffix after prefix")
if " " in branch_name:
issues.append("Branch name should not contain spaces (use hyphens instead)")
return issues
def analyze_commit_messages(self, mr_iid: int) -> List[str]:
"""Analyze commit message quality."""
commits_url = f"{self.gitlab_url}/api/v4/projects/{self.project_id}/merge_requests/{mr_iid}/commits"
commits = requests.get(commits_url, headers=self.headers).json()
issues = []
for commit in commits:
message = commit["message"]
# Check commit message format (Conventional Commits)
if not re.match(r'^(feat|fix|docs|style|refactor|test|chore)(\(.+\))?: .+', message):
issues.append(f"Commit {commit['short_id']}: Should follow Conventional Commits format")
# Check message length
first_line = message.split('\n')[0]
if len(first_line) > 72:
issues.append(f"Commit {commit['short_id']}: First line too long ({len(first_line)} chars)")
if len(first_line) < 10:
issues.append(f"Commit {commit['short_id']}: Message too short")
return issues
# GitLab CI integration
def main():
import os
if not os.getenv("CI_MERGE_REQUEST_IID"):
print("Not running in merge request context")
return
bot = GitLabCodeReviewBot(
gitlab_url=os.getenv("CI_API_V4_URL").replace("/api/v4", ""),
token=os.getenv("CI_JOB_TOKEN"),
project_id=int(os.getenv("CI_PROJECT_ID"))
)
mr_iid = int(os.getenv("CI_MERGE_REQUEST_IID"))
# Perform analysis
analysis = bot.analyze_merge_request(mr_iid)
# Check branch naming
branch_issues = bot.check_branch_naming(os.getenv("CI_MERGE_REQUEST_SOURCE_BRANCH_NAME"))
for issue in branch_issues:
analysis["issues"].append({
"type": "branch_naming",
"file": "N/A",
"message": issue
})
# Check commit messages
commit_issues = bot.analyze_commit_messages(mr_iid)
for issue in commit_issues:
analysis["issues"].append({
"type": "commit_message",
"file": "N/A",
"message": issue
})
# Post review
bot.post_review_comment(mr_iid, analysis)
# Exit with error code if issues found
if analysis["issues"]:
print(f"Found {len(analysis['issues'])} issues")
exit(1)
else:
print("No issues found")
if __name__ == "__main__":
main()
Troubleshooting Common Issues
Installation and Configuration Issues
Common GitLab Installation Problems
Issue | Symptoms | Solution | Prevention |
---|---|---|---|
Insufficient Memory | GitLab fails to start, 502 errors | Increase RAM to minimum 4GB | Monitor memory usage with free -h |
Port Conflicts | Cannot bind to port 80/443 | Stop conflicting services or change ports | Check ports with netstat -tuln |
SSL Certificate Issues | HTTPS not working, browser warnings | Verify certificate chain and permissions | Use Let’s Encrypt for automatic renewal |
Database Connection Failures | 500 errors, cannot connect to PostgreSQL | Check PostgreSQL service and credentials | Monitor with gitlab-ctl status |
Gitaly Storage Issues | Git operations fail, repository unavailable | Check Gitaly service and storage permissions | Monitor disk space regularly |
GitLab Troubleshooting Commands
# Complete GitLab health check
#!/bin/bash
echo "🔍 GitLab Health Check Starting..."
# Check GitLab services status
echo "📊 Service Status:"
sudo gitlab-ctl status
# Check configuration
echo "🔧 Configuration Check:"
sudo gitlab-ctl check-config
# Check GitLab application status
echo "🏥 Application Health:"
sudo gitlab-rake gitlab:check
# Check environment info
echo "💻 Environment Info:"
sudo gitlab-rake gitlab:env:info
# Check GitLab version
echo "📦 Version Information:"
sudo gitlab-rake gitlab:version
# Check database status
echo "🗄️ Database Status:"
sudo gitlab-psql -c "SELECT version();"
# Check Redis status
echo "⚡ Redis Status:"
sudo gitlab-redis-cli ping
# Check disk space
echo "💾 Disk Usage:"
df -h
# Check memory usage
echo "🧠 Memory Usage:"
free -h
# Check recent logs for errors
echo "📝 Recent Error Logs:"
sudo gitlab-ctl tail gitlab-workhorse | head -20
sudo gitlab-ctl tail postgresql | head -20
sudo gitlab-ctl tail redis | head -20
echo "✅ Health check completed!"
Performance Troubleshooting
# GitLab performance analysis script
#!/bin/bash
echo "🚀 GitLab Performance Analysis"
# Check Puma worker processes
echo "👷 Puma Workers:"
ps aux | grep puma | grep -v grep
# Check Sidekiq queues
echo "📬 Sidekiq Queue Status:"
sudo gitlab-rails runner "
queues = Sidekiq::Queue.all
queues.each do |queue|
puts \"#{queue.name}: #{queue.size} jobs\"
end
"
# Check slow queries
echo "🐌 Slow Database Queries:"
sudo gitlab-psql -c "
SELECT query, calls, total_time, mean_time, rows
FROM pg_stat_statements
ORDER BY total_time DESC
LIMIT 10;
"
# Check GitLab metrics
echo "📈 GitLab Metrics:"
curl -s "http://localhost:9090/-/metrics" | grep -E "(gitlab_database_|gitlab_cache_|gitlab_git_)"
# Check system resources
echo "💻 System Resources:"
echo "CPU Load: $(cat /proc/loadavg)"
echo "Memory Usage: $(free | grep Mem | awk '{printf \"%.2f%%\", $3/$2 * 100.0}')"
echo "Disk I/O: $(iostat -x 1 1 | tail -1)"
CI/CD Pipeline Debugging
Pipeline Failure Analysis
# Debug-friendly CI/CD configuration
stages:
- debug
- build
- test
variables:
# Enable debug mode
CI_DEBUG_TRACE: "true"
GITLAB_CI_DEBUG: "true"
# Debug information job
debug_info:
stage: debug
script:
- echo "🔍 Debug Information"
- echo "Pipeline ID: $CI_PIPELINE_ID"
- echo "Job ID: $CI_JOB_ID"
- echo "Runner: $CI_RUNNER_DESCRIPTION"
- echo "Branch: $CI_COMMIT_REF_NAME"
- echo "Commit: $CI_COMMIT_SHA"
- echo "Project: $CI_PROJECT_PATH"
- echo "Environment variables:"
- env | sort | grep CI_
- echo "System information:"
- uname -a
- cat /etc/os-release
- df -h
- free -h
artifacts:
paths:
- debug-info.txt
when: always
allow_failure: true
# Enhanced error handling
build_with_debug:
stage: build
script:
- set -e # Exit on any error
- set -x # Print commands being executed
- |
# Function for error handling
handle_error() {
echo "❌ Error occurred in build stage"
echo "Last command exit code: $?"
echo "Failed command: $BASH_COMMAND"
# Collect debug information
echo "Environment at failure:"
env | sort > error-env.txt
# Save build logs
if [ -f build.log ]; then
cp build.log error-build.log
fi
exit 1
}
# Set trap for error handling
trap 'handle_error' ERR
# Your build commands here
npm ci 2>&1 | tee build.log
npm run build 2>&1 | tee -a build.log
artifacts:
paths:
- error-*.txt
- error-*.log
when: on_failure
expire_in: 7 days
# Test with retry logic
test_with_retry:
stage: test
retry:
max: 3
when:
- runner_system_failure
- stuck_or_timeout_failure
- scheduler_failure
script:
- |
# Retry function for flaky tests
retry_command() {
local max_attempts=3
local attempt=1
local command="$@"
while [ $attempt -le $max_attempts ]; do
echo "Attempt $attempt of $max_attempts: $command"
if $command; then
echo "✅ Command succeeded on attempt $attempt"
return 0
else
echo "❌ Command failed on attempt $attempt"
if [ $attempt -lt $max_attempts ]; then
echo "Waiting 30 seconds before retry..."
sleep 30
fi
fi
attempt=$((attempt + 1))
done
echo "💥 Command failed after $max_attempts attempts"
return 1
}
# Run tests with retry
retry_command npm test
Common Pipeline Issues and Solutions
# Pipeline troubleshooting guide
#!/bin/bash
echo "🔧 GitLab CI/CD Troubleshooting Guide"
# Function to check runner availability
check_runners() {
echo "🏃 Checking GitLab Runners..."
# Get runners via API
curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
"$GITLAB_URL/api/v4/runners" | \
jq '.[] | {id, description, active, status}'
}
# Function to check runner logs
check_runner_logs() {
echo "📝 Recent Runner Logs:"
# For system service runners
if systemctl is-active --quiet gitlab-runner; then
journalctl -u gitlab-runner --since "1 hour ago" --no-pager
fi
# For Docker runners
if docker ps | grep -q gitlab-runner; then
docker logs $(docker ps | grep gitlab-runner | awk '{print $1}') --since 1h
fi
}
# Function to analyze failed jobs
analyze_failed_job() {
local job_id=$1
echo "🔍 Analyzing failed job: $job_id"
# Get job details
curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
"$GITLAB_URL/api/v4/projects/$PROJECT_ID/jobs/$job_id" | \
jq '{
status,
stage,
failure_reason,
runner: .runner.description,
started_at,
finished_at,
duration
}'
# Get job trace/logs
echo "📋 Job trace:"
curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
"$GITLAB_URL/api/v4/projects/$PROJECT_ID/jobs/$job_id/trace"
}
# Function to check resource usage
check_resources() {
echo "💻 System Resources:"
echo "CPU Usage:"
top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d'%' -f1
echo "Memory Usage:"
free | grep Mem | awk '{printf "%.2f%%\n", $3/$2 * 100.0}'
echo "Disk Usage:"
df -h | grep -E '^/dev'
echo "Docker Usage:"
if command -v docker >/dev/null 2>&1; then
docker system df
fi
}
# Main troubleshooting function
main() {
case "$1" in
runners)
check_runners
;;
logs)
check_runner_logs
;;
job)
analyze_failed_job "$2"
;;
resources)
check_resources
;;
*)
echo "Usage: $0 {runners|logs|job <job_id>|resources}"
echo "Examples:"
echo " $0 runners # Check runner status"
echo " $0 logs # View recent logs"
echo " $0 job 12345 # Analyze specific job"
echo " $0 resources # Check system resources"
;;
esac
}
main "$@"
Security and Access Issues
Common Security Problems
# Security troubleshooting script
#!/bin/bash
echo "🔒 GitLab Security Diagnostic"
# Check SSL/TLS configuration
check_ssl() {
echo "🔐 SSL/TLS Check:"
local domain="gitlab.example.com"
# Check certificate validity
echo "Certificate info:"
openssl s_client -connect $domain:443 -servername $domain < /dev/null 2>/dev/null | \
openssl x509 -text -noout | grep -E "(Subject:|Issuer:|Not Before:|Not After:)"
# Check SSL Labs rating (requires internet)
echo "SSL Labs rating:"
curl -s "https://api.ssllabs.com/api/v3/analyze?host=$domain" | \
jq '.endpoints[0].grade // "Not available"'
}
```bash
# Security troubleshooting script
#!/bin/bash
# Check user permissions
check_permissions() {
echo "👥 User Permissions Check:"
# List users with admin privileges
gitlab-rails runner "
User.admins.each do |user|
puts \"Admin: #{user.username} - #{user.email}\"
end
"
# Check for users with external authentication
gitlab-rails runner "
User.where.not(provider: nil).each do |user|
puts \"External user: #{user.username} - Provider: #{user.provider}\"
end
"
# Check for locked users
gitlab-rails runner "
User.locked.each do |user|
puts \"Locked user: #{user.username} - Locked at: #{user.locked_at}\"
end
"
}
# Check authentication configuration
check_auth_config() {
echo "🔑 Authentication Configuration:"
# Check LDAP configuration
if grep -q "ldap" /etc/gitlab/gitlab.rb; then
echo "LDAP configuration found"
grep -A 20 "gitlab_rails\['ldap_enabled'\]" /etc/gitlab/gitlab.rb
fi
# Check OAuth configuration
if grep -q "oauth" /etc/gitlab/gitlab.rb; then
echo "OAuth configuration found"
grep -A 10 "omniauth" /etc/gitlab/gitlab.rb
fi
# Check two-factor authentication status
gitlab-rails runner "
total_users = User.count
mfa_users = User.joins(:u2f_registrations).distinct.count
puts \"2FA Status: #{mfa_users}/#{total_users} users (#{(mfa_users.to_f/total_users*100).round(1)}%)\"
"
}
# Check for security vulnerabilities
check_vulnerabilities() {
echo "🛡️ Security Vulnerability Check:"
# Check GitLab version
current_version=$(gitlab-rake gitlab:version | grep "GitLab:" | awk '{print $2}')
echo "Current GitLab version: $current_version"
# Check for known vulnerabilities (would typically query security database)
echo "Checking for known vulnerabilities..."
# Check file permissions
echo "File permissions check:"
find /etc/gitlab -name "*.rb" -not -perm 600 -ls
find /var/opt/gitlab -name "*.log" -not -perm 640 -ls
}
# Check audit logs
check_audit_logs() {
echo "📋 Audit Log Analysis:"
# Check authentication failures
gitlab-rails runner "
puts 'Recent authentication failures:'
AuditEvent.where(details: {failed_login: true})
.where('created_at > ?', 24.hours.ago)
.limit(10).each do |event|
puts \"#{event.created_at}: Failed login attempt from #{event.ip_address}\"
end
"
# Check admin actions
gitlab-rails runner "
puts 'Recent admin actions:'
AuditEvent.joins(:user)
.where(users: {admin: true})
.where('audit_events.created_at > ?', 24.hours.ago)
.limit(10).each do |event|
puts \"#{event.created_at}: #{event.author.username} - #{event.details}\"
end
"
}
# Main security check function
security_main() {
case "$1" in
ssl)
check_ssl
;;
permissions)
check_permissions
;;
auth)
check_auth_config
;;
vulnerabilities)
check_vulnerabilities
;;
audit)
check_audit_logs
;;
all)
check_ssl
check_permissions
check_auth_config
check_vulnerabilities
check_audit_logs
;;
*)
echo "Usage: $0 {ssl|permissions|auth|vulnerabilities|audit|all}"
;;
esac
}
security_main "$@"
Access Control Troubleshooting
# Access control diagnostic script
#!/bin/bash
# Check project permissions
check_project_permissions() {
local project_id=$1
echo "🔐 Project Permissions for ID: $project_id"
gitlab-rails runner "
project = Project.find($project_id)
puts \"Project: #{project.full_name}\"
puts \"Visibility: #{project.visibility}\"
puts \"\"
puts 'Members:'
project.project_members.includes(:user).each do |member|
user = member.user
puts \" #{user.username} (#{user.email}) - Access Level: #{member.human_access}\"
end
puts \"\"
puts 'Group Members:'
if project.group
project.group.group_members.includes(:user).each do |member|
user = member.user
puts \" #{user.username} (#{user.email}) - Access Level: #{member.human_access}\"
end
end
"
}
# Check SSH key issues
check_ssh_keys() {
echo "🔑 SSH Key Diagnostics"
# Check SSH daemon configuration
echo "SSH daemon status:"
systemctl status ssh || systemctl status sshd
echo "GitLab Shell SSH configuration:"
cat /var/opt/gitlab/gitlab-shell/config.yml | grep -A 10 ssh
# Check authorized keys
echo "GitLab SSH authorized keys:"
if [ -f /var/opt/gitlab/.ssh/authorized_keys ]; then
wc -l /var/opt/gitlab/.ssh/authorized_keys
echo "Recent keys:"
tail -5 /var/opt/gitlab/.ssh/authorized_keys
fi
}
# Test repository access
test_repository_access() {
local repo_url=$1
local test_user=$2
echo "🧪 Testing repository access: $repo_url"
# Test HTTPS access
echo "Testing HTTPS access..."
git ls-remote "$repo_url" >/dev/null 2>&1 && echo "✅ HTTPS access OK" || echo "❌ HTTPS access failed"
# Test SSH access
echo "Testing SSH access..."
ssh_url=$(echo "$repo_url" | sed 's|https://[^/]*/|[email protected]:|')
git ls-remote "$ssh_url" >/dev/null 2>&1 && echo "✅ SSH access OK" || echo "❌ SSH access failed"
}
Performance Issues
Database Performance Optimization
-- Database performance analysis queries
-- Query to find slow queries
SELECT
query,
calls,
total_time / 1000.0 as total_time_seconds,
mean_time / 1000.0 as mean_time_seconds,
(total_time / calls) / 1000.0 as avg_time_seconds,
rows,
100.0 * shared_blks_hit / nullif(shared_blks_hit + shared_blks_read, 0) AS hit_percent
FROM pg_stat_statements
WHERE calls > 100
ORDER BY total_time DESC
LIMIT 20;
-- Index usage analysis
SELECT
schemaname,
tablename,
indexname,
idx_scan,
idx_tup_read,
idx_tup_fetch,
pg_size_pretty(pg_relation_size(indexname::regclass)) as index_size
FROM pg_stat_user_indexes
ORDER BY idx_scan DESC;
-- Table bloat analysis
SELECT
schemaname,
tablename,
pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) as total_size,
pg_size_pretty(pg_relation_size(schemaname||'.'||tablename)) as table_size,
pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename) - pg_relation_size(schemaname||'.'||tablename)) as index_size,
n_tup_ins as inserts,
n_tup_upd as updates,
n_tup_del as deletes,
n_live_tup as live_rows,
n_dead_tup as dead_rows
FROM pg_stat_user_tables
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC
LIMIT 20;
-- Connection analysis
SELECT
state,
count(*) as connections,
max(now() - state_change) as max_duration
FROM pg_stat_activity
WHERE pid <> pg_backend_pid()
GROUP BY state;
GitLab Performance Monitoring
#!/usr/bin/env python3
"""
GitLab Performance Monitor
Collects and analyzes GitLab performance metrics
"""
import requests
import json
import time
import psutil
import subprocess
from datetime import datetime
from typing import Dict, List
class GitLabPerformanceMonitor:
def __init__(self, gitlab_url: str, admin_token: str):
self.gitlab_url = gitlab_url
self.admin_token = admin_token
self.headers = {"Authorization": f"Bearer {admin_token}"}
def get_system_metrics(self) -> Dict:
"""Collect system performance metrics."""
return {
"timestamp": datetime.now().isoformat(),
"cpu_percent": psutil.cpu_percent(interval=1),
"memory_percent": psutil.virtual_memory().percent,
"disk_usage": {
"/": psutil.disk_usage("/").percent,
"/var/opt/gitlab": psutil.disk_usage("/var/opt/gitlab").percent if os.path.exists("/var/opt/gitlab") else None
},
"load_average": os.getloadavg(),
"network_io": psutil.net_io_counters()._asdict(),
"disk_io": psutil.disk_io_counters()._asdict()
}
def get_gitlab_metrics(self) -> Dict:
"""Collect GitLab application metrics."""
try:
response = requests.get(f"{self.gitlab_url}/-/metrics", timeout=10)
metrics_text = response.text
# Parse Prometheus metrics
metrics = {}
for line in metrics_text.split('\n'):
if line.startswith('gitlab_') and not line.startswith('#'):
parts = line.split(' ')
if len(parts) == 2:
metric_name = parts[0].split('{')[0]
metric_value = float(parts[1])
metrics[metric_name] = metric_value
return metrics
except Exception as e:
return {"error": str(e)}
def get_database_metrics(self) -> Dict:
"""Collect database performance metrics."""
try:
# Execute database queries via gitlab-rails
queries = {
"active_connections": "SELECT count(*) FROM pg_stat_activity WHERE state = 'active'",
"database_size": "SELECT pg_size_pretty(pg_database_size(current_database()))",
"slow_queries": "SELECT count(*) FROM pg_stat_statements WHERE mean_time > 1000",
"cache_hit_ratio": """
SELECT round(
100.0 * sum(blks_hit) / nullif(sum(blks_hit) + sum(blks_read), 0), 2
) FROM pg_stat_database WHERE datname = current_database()
"""
}
results = {}
for name, query in queries.items():
try:
result = subprocess.run([
'sudo', 'gitlab-psql', '-t', '-c', query
], capture_output=True, text=True, timeout=30)
if result.returncode == 0:
results[name] = result.stdout.strip()
else:
results[name] = f"Error: {result.stderr}"
except subprocess.TimeoutExpired:
results[name] = "Timeout"
except Exception as e:
results[name] = f"Error: {str(e)}"
return results
except Exception as e:
return {"error": str(e)}
def get_sidekiq_metrics(self) -> Dict:
"""Collect Sidekiq job queue metrics."""
try:
result = subprocess.run([
'sudo', 'gitlab-rails', 'runner', '''
require "sidekiq/api"
stats = Sidekiq::Stats.new
puts JSON.generate({
processed: stats.processed,
failed: stats.failed,
enqueued: stats.enqueued,
scheduled: stats.scheduled_size,
retry_size: stats.retry_size,
dead_size: stats.dead_size,
processes: stats.processes_size,
default_queue_size: Sidekiq::Queue.new.size,
queues: Sidekiq::Queue.all.map { |q| {q.name => q.size} }.reduce({}, :merge)
})
'''
], capture_output=True, text=True, timeout=60)
if result.returncode == 0:
return json.loads(result.stdout)
else:
return {"error": result.stderr}
except Exception as e:
return {"error": str(e)}
def get_git_metrics(self) -> Dict:
"""Collect Git operation metrics."""
try:
result = subprocess.run([
'sudo', 'gitlab-rails', 'runner', '''
puts JSON.generate({
total_repositories: Project.count,
total_commits: Project.joins(:repository).sum(:commit_count),
total_branches: Project.joins(:repository).sum("repository_size"),
storage_usage: Project.sum(:repository_size)
})
'''
], capture_output=True, text=True, timeout=60)
if result.returncode == 0:
return json.loads(result.stdout)
else:
return {"error": result.stderr}
except Exception as e:
return {"error": str(e)}
def generate_performance_report(self) -> Dict:
"""Generate comprehensive performance report."""
print("📊 Collecting GitLab performance metrics...")
report = {
"timestamp": datetime.now().isoformat(),
"system": self.get_system_metrics(),
"gitlab": self.get_gitlab_metrics(),
"database": self.get_database_metrics(),
"sidekiq": self.get_sidekiq_metrics(),
"git": self.get_git_metrics()
}
# Analyze and add recommendations
recommendations = self.analyze_performance(report)
report["recommendations"] = recommendations
return report
def analyze_performance(self, report: Dict) -> List[str]:
"""Analyze metrics and provide recommendations."""
recommendations = []
# System analysis
if report["system"]["cpu_percent"] > 80:
recommendations.append("High CPU usage detected. Consider scaling horizontally or optimizing workloads.")
if report["system"]["memory_percent"] > 85:
recommendations.append("High memory usage detected. Consider increasing RAM or optimizing memory usage.")
# Database analysis
try:
active_connections = int(report["database"]["active_connections"])
if active_connections > 100:
recommendations.append(f"High number of active database connections ({active_connections}). Consider connection pooling optimization.")
except:
pass
# Sidekiq analysis
sidekiq_data = report.get("sidekiq", {})
if isinstance(sidekiq_data, dict):
enqueued = sidekiq_data.get("enqueued", 0)
if enqueued > 1000:
recommendations.append(f"Large Sidekiq queue detected ({enqueued} jobs). Consider adding more workers.")
failed = sidekiq_data.get("failed", 0)
if failed > 100:
recommendations.append(f"Many failed Sidekiq jobs ({failed}). Review job failures and fix issues.")
if not recommendations:
recommendations.append("Performance metrics look healthy!")
return recommendations
def main():
import os
monitor = GitLabPerformanceMonitor(
gitlab_url=os.getenv("GITLAB_URL", "https://gitlab.example.com"),
admin_token=os.getenv("GITLAB_ADMIN_TOKEN", "")
)
# Generate performance report
report = monitor.generate_performance_report()
# Save report
with open(f"performance-report-{datetime.now().strftime('%Y%m%d-%H%M%S')}.json", "w") as f:
json.dump(report, f, indent=2)
# Print summary
print("\n📈 Performance Summary:")
print(f" CPU Usage: {report['system']['cpu_percent']:.1f}%")
print(f" Memory Usage: {report['system']['memory_percent']:.1f}%")
print(f" Database Connections: {report['database'].get('active_connections', 'N/A')}")
sidekiq_data = report.get("sidekiq", {})
if isinstance(sidekiq_data, dict):
print(f" Sidekiq Queue: {sidekiq_data.get('enqueued', 'N/A')} jobs")
print("\n💡 Recommendations:")
for rec in report["recommendations"]:
print(f" • {rec}")
if __name__ == "__main__":
main()
Frequently Asked Questions
General Questions
Q: What’s the main difference between GitLab Community Edition and Enterprise Edition?
A: GitLab Community Edition (CE) is open-source and free, providing core Git functionality, basic CI/CD, and project management. Enterprise Edition (EE) adds advanced features like:
- Advanced security scanning (SAST, DAST, dependency scanning)
- Compliance management and audit events
- Advanced authentication (LDAP, SAML, multi-factor)
- Geo-replication for distributed teams
- Advanced analytics and reporting
- Premium support
Q: Can I migrate from GitLab CE to EE without losing data?
A: Yes, GitLab EE is built on the same codebase as CE with additional features. Migration is straightforward:
- Stop GitLab services:
sudo gitlab-ctl stop
- Install GitLab EE package
- Run reconfigure:
sudo gitlab-ctl reconfigure
- Start services:
sudo gitlab-ctl start
All data, including repositories, issues, and settings, will be preserved.
Q: How does GitLab pricing compare to competitors?
A: GitLab offers competitive pricing with an all-in-one approach:
- Free tier: Unlimited private repositories, 400 CI/CD minutes
- Premium: $19/user/month - Advanced CI/CD, security scanning
- Ultimate: $99/user/month - Full DevSecOps platform
Compared to using separate tools (GitHub + Jenkins + security tools), GitLab often provides better ROI through consolidation.
Technical Questions
Q: What are the minimum system requirements for self-hosted GitLab?
A: Minimum requirements depend on user count:
Users | RAM | CPU Cores | Storage |
---|---|---|---|
1-100 | 4GB | 2 cores | 50GB |
100-500 | 8GB | 4 cores | 100GB |
500-1000 | 16GB | 8 cores | 200GB |
1000+ | 32GB+ | 16+ cores | 500GB+ |
For production environments, always exceed minimums and implement monitoring.
Q: How do I backup and restore GitLab?
A: GitLab provides built-in backup functionality:
# Create backup
sudo gitlab-backup create
# Restore from backup (stop GitLab first)
sudo gitlab-ctl stop puma
sudo gitlab-ctl stop sidekiq
sudo gitlab-backup restore BACKUP=1691234567_2023_08_05_16.2.0
# Restart services
sudo gitlab-ctl restart
sudo gitlab-rake gitlab:check SANITIZE=true
Q: Can GitLab integrate with existing tools?
A: Yes, GitLab offers extensive integration capabilities:
- APIs: Comprehensive REST API for custom integrations
- Webhooks: Real-time notifications to external systems
- Third-party integrations: Jira, Slack, Microsoft Teams, etc.
- SSO: LDAP, SAML, OAuth providers
- Monitoring: Prometheus, Grafana, external APM tools
CI/CD Questions
Q: How do GitLab Runners work and how many do I need?
A: GitLab Runners execute CI/CD jobs. Types include:
- Shared runners: Provided by GitLab.com or admin-managed
- Group runners: Available to all projects in a group
- Project runners: Dedicated to specific projects
Runner count depends on:
- Concurrent job requirements
- Job duration and frequency
- Resource availability
- Isolation needs
A good starting point is 1 runner per 10-20 active developers.
Q: Can I use my existing Jenkins pipelines with GitLab?
A: While you can’t directly import Jenkins pipelines, migration strategies include:
- Manual conversion: Translate Jenkinsfile to
.gitlab-ci.yml
- Wrapper approach: Call Jenkins jobs from GitLab CI
- Gradual migration: Move projects incrementally
- Auto-conversion tools: Third-party tools to assist migration
GitLab CI/CD often simplifies complex Jenkins pipelines due to tighter integration.
Q: How do I handle secrets and environment variables?
A: GitLab provides multiple secret management options:
# Project-level variables (Settings > CI/CD > Variables)
variables:
API_KEY: $PROJECT_API_KEY # Masked variable
DATABASE_URL: $DB_URL # Protected variable
# Group-level variables (inherited by all projects)
# File-type variables for certificates/keys
# External secret management integration (HashiCorp Vault, etc.)
Best practices:
- Use masked variables for sensitive data
- Implement protected variables for production secrets
- Consider external secret management for enterprise needs
- Regularly rotate secrets and audit access
Security Questions
Q: How secure is GitLab for enterprise use?
A: GitLab implements enterprise-grade security:
- Encryption: Data at rest and in transit
- Authentication: Multi-factor, SSO, LDAP integration
- Authorization: Role-based access control
- Compliance: SOC 2, ISO 27001 certifications
- Security scanning: Built-in vulnerability detection
- Audit logs: Comprehensive activity tracking
- Regular updates: Monthly security releases
For additional security, consider:
- Self-hosting for complete data control
- Network isolation and VPNs
- Regular security assessments
- Compliance-specific configurations
Q: What security scanning capabilities does GitLab provide?
A: GitLab Ultimate includes comprehensive DevSecOps features:
Scan Type | Purpose | Implementation |
---|---|---|
SAST | Static code analysis | Automatic detection of code vulnerabilities |
DAST | Dynamic application testing | Runtime vulnerability scanning |
Dependency Scanning | Third-party package vulnerabilities | Checks for known CVEs in dependencies |
Container Scanning | Docker image vulnerabilities | Scans container images for security issues |
Secret Detection | Exposed secrets in code | Detects API keys, passwords in commits |
License Compliance | License compatibility checking | Ensures license compatibility |
Q: How do I implement compliance requirements (SOX, HIPAA, etc.)?
A: GitLab supports compliance through:
# Compliance pipeline example
include:
- template: Security/SAST.gitlab-ci.yml
- template: Security/Dependency-Scanning.gitlab-ci.yml
compliance_check:
stage: compliance
script:
- audit-generator --standard SOX --project-id $CI_PROJECT_ID
- compliance-validator --config .compliance.yml
artifacts:
reports:
junit: compliance-report.xml
expire_in: 7 years # SOX retention requirement
Additional compliance features:
- Immutable audit logs
- Access control and segregation of duties
- Automated compliance reporting
- Change management workflows
- Data retention policies
Conclusion
GitLab represents a paradigm shift in software development tooling, moving away from the traditional “best-of-breed” approach toward an integrated, single-application DevOps platform. Throughout this comprehensive guide, we’ve explored how GitLab can transform your development workflow, enhance security posture, and streamline operations from code commit to production deployment.
Key Takeaways
1. Unified Platform Benefits GitLab’s greatest strength lies in its integration. By consolidating source code management, CI/CD, security scanning, project management, and monitoring into a single platform, teams experience:
- 40% reduction in context switching
- 200% improvement in deployment frequency
- 85% reduction in security vulnerabilities reaching production
- Simplified toolchain management and reduced vendor fatigue
2. Flexible Deployment Options Whether you choose GitLab.com’s SaaS offering or self-host with Community/Enterprise Edition, GitLab adapts to your organization’s needs:
- Startups: Benefit from generous free tiers and rapid scaling
- Enterprises: Gain complete control with self-hosting and advanced compliance features
- Hybrid organizations: Mix SaaS and self-hosted instances as needed
3. DevSecOps by Design GitLab’s “shift-left” security approach integrates protection throughout the development lifecycle:
- Automated security scanning catches 70% of vulnerabilities before production
- Built-in compliance frameworks support SOX, HIPAA, and PCI-DSS requirements
- Zero-configuration security scanning reduces security team bottlenecks
4. Scalability and Performance From small teams to Fortune 500 enterprises, GitLab scales through:
- Horizontal scaling with load balancing and database replication
- Kubernetes-native deployment options
- Auto-scaling CI/CD runners
- Global Geo-replication for distributed teams
Implementation Roadmap
For organizations considering GitLab adoption, we recommend this phased approach:
Phase 1: Foundation (Weeks 1-4)
- Migrate core repositories
- Set up basic CI/CD pipelines
- Train team on GitLab workflows
- Establish branch protection and merge request processes
Phase 2: Integration (Weeks 5-8)
- Implement security scanning
- Set up automated deployments
- Configure monitoring and alerting
- Integrate with existing tools (LDAP, Slack, etc.)
Phase 3: Optimization (Weeks 9-12)
- Fine-tune performance settings
- Implement advanced DevSecOps workflows
- Set up comprehensive monitoring
- Establish governance and compliance processes
Phase 4: Advanced Features (Month 4+)
- Deploy Geo-replication for global teams
- Implement advanced analytics and reporting
- Set up disaster recovery procedures
- Optimize for enterprise-scale operations
Future Considerations
As GitLab continues evolving, several trends will impact its adoption:
1. AI/ML Integration GitLab is investing heavily in AI-powered features:
- Intelligent code suggestions and review assistance
- Automated security vulnerability remediation
- Predictive analytics for deployment success
- Smart resource allocation for CI/CD jobs
2. Cloud-Native Evolution Kubernetes-first approach becoming standard:
- Native GitLab Kubernetes integration
- Serverless CI/CD execution
- Multi-cloud deployment strategies
- Container-native security scanning
3. Compliance Automation Enhanced regulatory compliance support:
- Automated compliance reporting
- Real-time policy enforcement
- Blockchain-based audit trails
- Industry-specific compliance templates
Final Recommendations
Choose GitLab if:
- You want to consolidate your DevOps toolchain
- Security integration is a priority
- Your team values self-hosting flexibility
- You need comprehensive project management features
- You’re implementing DevSecOps practices
Consider alternatives if:
- You’re heavily invested in existing tool ecosystems
- Your needs are primarily source code management
- You require specific third-party integrations not available in GitLab
- Your team prefers specialized best-of-breed tools
Getting Started Today
Ready to begin your GitLab journey? Start with these immediate steps:
- Evaluate your current toolchain - Document existing tools and integration points
- Set up a pilot project - Choose a non-critical project for initial testing
- Create a GitLab.com account - Explore features with the generous free tier
- Run the migration scripts provided in this guide for your current VCS
- Join the GitLab community - Engage with forums, documentation, and training resources
GitLab’s comprehensive approach to DevOps represents the future of software development platforms. By embracing its integrated philosophy, organizations can achieve faster delivery cycles, improved security postures, and more efficient development workflows.
The journey from traditional, fragmented toolchains to GitLab’s unified platform requires commitment and change management, but the results speak for themselves: teams report significantly improved productivity, reduced complexity, and enhanced collaboration. As the software development landscape continues evolving toward DevSecOps and cloud-native practices, GitLab positions organizations at the forefront of these technological advances.
Whether you’re a startup seeking rapid growth capabilities or an enterprise requiring robust compliance and scaling features, GitLab provides the foundation for modern software development excellence. The investment in learning and implementing GitLab pays dividends through improved developer experience, enhanced security, and streamlined operations that adapt to your organization’s changing needs.
This guide represents a comprehensive overview of GitLab’s capabilities as of August 2025. For the most current information on features, pricing, and technical specifications, always refer to GitLab’s official documentation and release notes.
Additional Resources: