AI Scanner (0din-ai/ai-scanner) Skill by ara.so — Daily 2026 Skills collection. AI Scanner is an open-source Ruby on Rails web application for AI model security assessments, wrapping NVIDIA garak with a multi-tenant UI, scheduling, PDF reports, and SIEM integration. It runs 179 community probes across 35 vulnerability families aligned with the OWASP LLM Top 10. Installation Quick Install (Docker) curl -sL https://raw.githubusercontent.com/0din-ai/ai-scanner/main/scripts/install.sh | bash Manual Install curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/dist/docker-compose.yml curl -O https://raw.githubusercontent.com/0din-ai/ai-scanner/main/.env.example cp .env.example .env Edit .env with required values:

Generate a secure key

openssl rand -hex 64

.env minimum required values

SECRET_KEY_BASE

< output_of_above_command

POSTGRES_PASSWORD

< your_secure_db_password

docker compose up -d Access at http://localhost — default credentials: admin@example.com / password . Change the default password immediately after first login. Configuration (.env)

Required

SECRET_KEY_BASE

< 64 -byte-hex

POSTGRES_PASSWORD

< strong-password

Optional: custom port

PORT

8080

Optional: SIEM integration

SPLUNK_HEC_URL

https://splunk.example.com:8088/services/collector SPLUNK_HEC_TOKEN = $SPLUNK_HEC_TOKEN RSYSLOG_HOST = syslog.example.com RSYSLOG_PORT = 514

Optional: email

SMTP_HOST

smtp.example.com SMTP_PORT = 587 SMTP_USERNAME = $SMTP_USERNAME SMTP_PASSWORD = $SMTP_PASSWORD Core Concepts Concept Description Target An AI system to test — API-based LLM or browser-based chat UI Probe A single attack test (e.g., prompt injection, data leakage) Scan A run of selected probes against a target ASR Attack Success Rate — percentage of probes that succeeded Organization Tenant boundary; users and scans are scoped per org Setting Up a Target Targets define what you're scanning. Two types: API-based LLM Target (e.g., OpenAI-compatible endpoint):

In Rails console or via UI — representative model

target

Target . create ! ( name : "Production GPT-4" , target_type : "api" , api_endpoint : "https://api.openai.com/v1/chat/completions" , api_key : ENV [ "OPENAI_API_KEY" ] , model_name : "gpt-4" , organization : current_organization ) Browser-based Chat UI Target : target = Target . create ! ( name : "Internal Chatbot UI" , target_type : "browser" , url : "https://chatbot.internal.example.com" , organization : current_organization ) Running a Scan Via UI Navigate to Targets → select your target Click New Scan Select probe families or individual probes Click Run Scan Via Rails Console

On-demand scan with specific probe families

scan

Scan . create ! ( target : target , probe_families : [ "prompt_injection" , "data_leakage" , "insecure_output" ] , organization : current_organization ) ScanJob . perform_later ( scan . id ) Scheduled Recurring Scan

Weekly scan every Monday at 2am

scheduled_scan

ScheduledScan . create ! ( target : target , probe_families : [ "prompt_injection" , "jailbreak" ] , cron_expression : "0 2 * * 1" , organization : current_organization ) Probe Families (35 total, aligned to OWASP LLM Top 10) Key probe families available:

List all available probe families

Garak :: ProbeRegistry . families

=> ["prompt_injection", "jailbreak", "data_leakage", "insecure_output",

"supply_chain", "sensitive_info", "excessive_agency", "overreliance",

"model_theft", "malicious_plugins", ...]

Get probes within a family

Garak :: ProbeRegistry . probes_for ( "prompt_injection" )

=> 179 total probes across all families

Viewing Results Attack Success Rate (ASR) scan = Scan . find ( scan_id ) puts scan . asr_score

=> 0.23 (23% attack success rate)

puts scan . status

=> "completed"

puts scan . probe_results . count

=> 47

Per-probe breakdown

scan . probe_results . each do | result | puts "

{

result . probe_name } :

{

result . passed ? ? 'SAFE' : 'VULNERABLE' } " puts " Attempts:

{

result . attempt_count } " puts " ASR:

{

result . asr_score } " end Trend Tracking

Compare ASR across scan runs for a target

target . scans . completed . order ( :created_at ) . map do | scan | { date : scan . created_at , asr : scan . asr_score } end PDF Report Export

Generate PDF report for a scan

scan

Scan . find ( scan_id ) pdf_path = ReportExporter . export_pdf ( scan )

Includes: executive summary, per-probe results, per-attempt drill-down

Via UI: Navigate to a completed scan → Export PDF . SIEM Integration Splunk

config/initializers/siem.rb

SiemIntegration . configure do | config | config . provider = :splunk config . splunk_hec_url = ENV [ "SPLUNK_HEC_URL" ] config . splunk_hec_token = ENV [ "SPLUNK_HEC_TOKEN" ] config . forward_on_completion = true end Rsyslog SiemIntegration . configure do | config | config . provider = :rsyslog config . rsyslog_host = ENV [ "RSYSLOG_HOST" ] config . rsyslog_port = ENV [ "RSYSLOG_PORT" ] . to_i end Multi-Tenant Organization Management

Create a new organization

org

Organization . create ! ( name : "Security Team Alpha" )

Invite a user

user

User . invite ! ( email : "analyst@example.com" , organization : org , role : "analyst"

roles: "admin", "analyst", "viewer"

)

Data is encrypted at rest per organization

org . encryption_key

=> managed automatically

Development Setup git clone https://github.com/0din-ai/ai-scanner.git cd ai-scanner cp .env.example .env.development

Install dependencies

bundle install

Database setup

rails db:create db:migrate db:seed

Install garak (Python dependency)

pip install garak

Start development server

bin/dev Running Tests

Full test suite

bundle exec rspec

Specific area

bundle exec rspec spec/models/scan_spec.rb bundle exec rspec spec/jobs/scan_job_spec.rb

Lint

bundle exec rubocop Common Patterns Testing a New LLM Before Deployment

Comprehensive pre-deployment scan

target

Target . create ! ( name : "New Model v2 - Pre-deploy" , target_type : "api" , api_endpoint : ENV [ "NEW_MODEL_ENDPOINT" ] , api_key : ENV [ "NEW_MODEL_API_KEY" ] , model_name : "new-model-v2" , organization : current_organization )

Run all 35 probe families

scan

Scan . create ! ( target : target , probe_families : Garak :: ProbeRegistry . families , organization : current_organization ) ScanJob . perform_now ( scan . id ) if scan . reload . asr_score

0.15 puts "WARNING: ASR

{

scan . asr_score } exceeds threshold. Review before deploying." else puts "PASS: Model meets security threshold." end Using the Mock LLM for Testing Scanner Setup The built-in Mock LLM lets you validate your scanner configuration without hitting real APIs: target = Target . create ! ( name : "Mock LLM" , target_type : "mock" , organization : current_organization )

Run a quick scan to verify everything works end-to-end

Webhook on Scan Completion

config/initializers/scan_hooks.rb

ActiveSupport :: Notifications . subscribe ( "scan.completed" ) do | _ , _ , _ , _ , payload | scan = Scan . find ( payload [ :scan_id ] ) if scan . asr_score

0.20 SlackNotifier . alert ( channel : "#security-alerts" , message : "High ASR detected:

{

scan . asr_score } on

{

scan . target . name } " ) end end Docker Compose Production Tips

docker-compose.override.yml — production additions

services : web : environment : RAILS_ENV : production FORCE_SSL : "true" labels : - "traefik.enable=true" - "traefik.http.routers.scanner.rule=Host(scanner.example.com)" - "traefik.http.routers.scanner.tls.certresolver=letsencrypt"

Upgrade

docker compose pull docker compose up -d docker compose exec web rails db:migrate Troubleshooting Problem Solution Scan stuck in "running" Check docker compose logs worker — garak Python process may have crashed SECRET_KEY_BASE error on start Run openssl rand -hex 64 and set in .env Can't connect to target API Verify API key env var is set; check firewall allows outbound from container Browser target scan fails Ensure Playwright/Chrome is available in the worker container PDF export blank Check wkhtmltopdf is installed in the web container SIEM not receiving events Verify SPLUNK_HEC_URL includes full path /services/collector

View all service logs

docker compose logs -f

Check worker specifically (runs garak)

docker compose logs -f worker

Rails console for debugging

docker compose exec web rails console

Check garak is working

docker compose exec worker python -c "import garak; print(garak.version)" Key Files (for Contributors) app/ models/ scan.rb # Core scan model, ASR calculation target.rb # Target types and validation probe_result.rb # Per-probe result storage jobs/ scan_job.rb # Async job that invokes garak services/ garak_runner.rb # Ruby wrapper around garak CLI report_exporter.rb siem_integration.rb lib/ garak/ probe_registry.rb # 179 probes, 35 families dist/ docker-compose.yml # Production compose file scripts/ install.sh # One-line installer Resources Full Documentation Quick Start Guide NVIDIA garak OWASP LLM Top 10 CONTRIBUTING.md SECURITY.md