- Twitter Search and Analysis
- Overview
- Search Twitter for keywords using advanced search syntax, fetch up to 1000 relevant tweets, and analyze the data to produce professional reports with insights, statistics, and actionable recommendations.
- Prerequisites
- API Key Required
-
- Users must configure their Twitter API key from
- https://twitterapi.io
- The API key can be provided in three ways:
- Environment variable
- (recommended): Set
- TWITTER_API_KEY
- in your
- ~/.bashrc
- or
- ~/.zshrc
- echo
- 'export TWITTER_API_KEY="your_key_here"'
- >>
- ~/.bashrc
- source
- ~/.bashrc
- As an argument
-
- Use
- --api-key YOUR_KEY
- with the wrapper script
- Passed directly
- As first argument to the Python script Quick Start Using the Wrapper Script (Recommended) The wrapper script automatically handles environment variable loading and dependency checks:
Basic search (uses TWITTER_API_KEY from shell config)
./scripts/run_search.sh "AI"
With custom API key
./scripts/run_search.sh "AI" --api-key YOUR_KEY
With options
./scripts/run_search.sh " \" Claude AI \" " --max-results 100 --format summary
Advanced query
./scripts/run_search.sh "from:elonmusk since:2024-01-01" --query-type Latest Direct Python Script Usage
Search for a keyword
scripts/twitter_search.py " $API_KEY " "AI"
Search with multiple keywords
scripts/twitter_search.py " $API_KEY " " \" ChatGPT \" OR \" Claude AI \" "
Search from specific user
scripts/twitter_search.py " $API_KEY " "from:elonmusk"
Search with date range
scripts/twitter_search.py " $API_KEY " "Bitcoin since:2024-01-01" Advanced Queries
Complex query: AI tweets from verified users, English only
scripts/twitter_search.py " $API_KEY " "AI OR \" machine learning \" lang:en filter:verified"
Recent crypto tweets with minimum engagement
scripts/twitter_search.py " $API_KEY " "Bitcoin min_retweets:10 lang:en"
From specific influencers
scripts/twitter_search.py " $API_KEY " "from:elonmusk OR from:VitalikButerin since:2024-01-01" Output Format
Full JSON with all tweets
scripts/twitter_search.py " $API_KEY " "AI" --format json
Summary with statistics (default)
- scripts/twitter_search.py
- "
- $API_KEY
- "
- "AI"
- --format
- summary
- Options
- --max-results N
-
- Maximum tweets to fetch (default: 1000)
- --query-type Latest|Top
-
- Sort order (default: Top for relevance)
- --format json|summary
- Output format (default: summary) Workflow 1. Understand User Requirements Clarify the analysis goal: What topic/keyword to search? Date range preference? Specific users to include/exclude? Language preference? Type of insights needed (trends, sentiment, influencers)? 2. Build the Search Query Use Twitter Advanced Search syntax: Syntax Example Description keyword AI Single keyword "phrase" "machine learning" Exact phrase OR AI OR ChatGPT Either term from:user from:elonmusk From specific user to:user to:elonmusk Reply to user since:DATE since:2024-01-01 After date until:DATE until:2024-12-31 Before date lang:xx lang:en Language code
hashtag
AI
- Hashtag
- filter:links
- filter:links
- Tweets with links
- min_retweets:N
- min_retweets:100
- Minimum retweets
- 3. Fetch Data
- Execute the search script:
- scripts/twitter_search.py
- "
- $API_KEY
- "
- "YOUR_QUERY"
- --max-results
- 1000
- --query-type Top
- Important
-
- Default is 1000 tweets maximum. The script automatically:
- Paginates through all available results
- Stops at 1000 tweets (API limit consideration)
- Handles errors gracefully
- 4. Analyze and Generate Report
- After fetching data, produce a comprehensive professional report with:
- Report Structure
- Executive Summary
- (2-3 sentences)
- What was searched
- Key findings overview
- Data Overview
- Total tweets analyzed
- Date range of data
- Query parameters used
- Key Metrics
- Total engagement (likes, retweets, replies, quotes, views)
- Average engagement per tweet
- Language distribution
- Reply vs. original tweet ratio
- Top Content Analysis
- Most retweeted tweets (with
- URL links
- to original tweets)
- Most liked tweets (with
- URL links
- to original tweets)
- Top hashtags with frequency
- Most mentioned users
- Selected tweet examples with full URL references
- Influencer Analysis
- Top users by follower count
- Most active users
- Verified user percentage
- Trend Insights
- (based on data patterns)
- Emerging themes
- Sentiment indicators
- Temporal patterns
- Conversation drivers
- Key Takeaways
- 3-5 bullet points of core insights
- Data-backed conclusions
- Actionable Recommendations
- Specific, implementable suggestions
- Based on the data findings
- Prioritized by impact
- Analysis Guidelines
- Be data-driven
-
- Every claim should reference actual metrics
- Provide context
-
- Explain why metrics matter
- Identify patterns
-
- Look for trends across the dataset
- Stay objective
-
- Present facts, avoid speculation
- Be specific
-
- Recommendations should be concrete and actionable
- Consider external context
- Use web search for background when relevant 5. Output Format Present the report in clear markdown with: Headers for each section Tables for structured data Bullet points for lists Bold for key metrics Code blocks for tweet examples Clickable URLs for all referenced tweets (format: @username ) Tweet URL Format Always include clickable links to tweets: | Author | Tweet | URL | |
|
|
- |
- |
- @user
- |
- Summary of tweet content
- |
- [
- View
- ](
- https://x.com/user/status/123456
- )
- |
- Or inline format:
- -
- **
- @username
- **
- Tweet summary - View Tweet Query Examples by Use Case Trend Analysis "AI" OR "artificial intelligence" lang:en min_retweets:50 Competitor Monitoring from:competitor1 OR from:competitor2 since:2024-01-01 Product Launch Tracking
ProductName OR "Product Name" lang:en filter:verified
Crisis Monitoring
BrandName OR "Brand Name" lang:en --query-type Latest
Influencer Discovery
Topic lang:en min_retweets:100 min_faves:500
- Sentiment Analysis
- "brand name" OR #BrandName lang:en --max-results 1000
- Resources
- scripts/run_search.sh (Wrapper Script)
- Convenience wrapper that handles environment variable loading and dependency checks:
- Automatically loads
- TWITTER_API_KEY
- from
- ~/.bashrc
- or
- ~/.zshrc
- Checks Python availability and installs missing dependencies
- Provides user-friendly error messages
- Supports all command-line options from the Python script
- Usage
- :
- ./scripts/run_search.sh
- <
- query
- >
- [
- options
- ]
- Options
- :
- --api-key KEY
-
- Override environment variable API key
- --max-results N
-
- Maximum tweets to fetch (default: 1000)
- --query-type Latest|Top
-
- Sort order (default: Top)
- --format json|summary
-
- Output format (default: json)
- scripts/twitter_search.py
- Executable Python script that:
- Fetches tweets from Twitter API
- Handles pagination automatically
- Extracts key tweet metrics
- Calculates aggregate statistics
- Outputs structured JSON data
- Usage
- :
- scripts/twitter_search.py
- <
- api_key
- >
- <
- query
- >
- [
- options
- ]
- references/twitter_api.md
- Comprehensive API documentation including:
- Complete parameter reference
- Query syntax guide
- Response structure details
- Pagination instructions
- Best practices for analysis
- Error handling guide
- Read this when
- Building complex queries or understanding data structure. Tips for Better Analysis Use Top query type for trend analysis (more relevant results) Set date filters for timely insights Filter by language for accurate text analysis Include minimum engagement to filter noise Combine with web search to validate trends Look beyond metrics - analyze content themes Track hashtags to identify sub-conversations Identify influencers by combining followers + engagement Error Handling If the script fails: Check API key validity Verify query syntax Ensure network connectivity Check rate limits (if applicable) Review error messages for specific issues