google-flight-search

Evaluation

Total Score: 7/10

Evaluation Summary

• Strengths:
• 1. Accuracy/Correctness:
• • The agent generally provides detailed and accurate flight options, including airlines, departure and arrival times, layovers, total durations, prices, and additional features like legroom and carbon emissions.
• 2. Completeness:
• • Multiple flight options are presented for most queries, offering a range of choices for the user.
• • Each option is detailed, providing comprehensive information for decision-making.
• 3. Consistency with Claimed Capabilities:
• • The agent consistently aligns with its claimed capability of assisting with flight searches using the Google Flights search tool.
• 4. Usefulness / Clarity:
• • The answers are well-structured, making it easy for users to compare different flight options.
• • The inclusion of Google Flights links allows users to explore more options if needed.
• Weaknesses:
• 1. Time Relevance:
• • The agent does not specify when the flight information was retrieved, which is crucial given the rapidly changing nature of flight prices and availability.
• 2. Currency of Data:
• • Without a timestamp, the currency of the flight information is unclear, potentially leading to discrepancies if users check the options later.
• 3. Handling Errors:
• • The agent occasionally fails to provide answers, as seen in the query for flights from Chicago to London, indicating limitations in data retrieval or processing.
• Suggestions for Improvement:
• 1. Include a Timestamp:
• • Clearly state when the flight information was retrieved to provide context on its currency.
• 2. Advise on Potential Changes:
• • Inform users that flight prices and availability are subject to change and recommend checking the provided links for the most up-to-date information.
• 3. Improve Error Handling:
• • Enhance the agent's ability to handle errors and provide alternative solutions or explanations when data retrieval fails.
• Overall, the agent performs well in providing detailed and accurate flight information but needs improvements in time relevance and error handling.

• Test case 1

  Score: 8/10

  Find a round-trip flight from New York City to Tokyo, departing on September 15, 2025, and returning on September 29, 2025, for two adults in business class. Provide the best available options based on price and duration.

  Here are the best available round-trip flight options from New York City (JFK) to Tokyo (Narita International Airport - NRT) for two adults in business class, departing on September 15, 2025, and returning on September 29, 2025:

  1. Flight Option 1

     • Airline: WestJet
     • Departure: JFK on September 15, 2025, at 17:30
     • Arrival: Calgary International Airport (YYC) on September 15, 2025, at 21:03
     • Layover: Calgary (YYC) for about 18 hours (overnight)
     • Departure from Calgary: September 16, 2025, at 15:05
     • Arrival in Tokyo: NRT on September 17, 2025, at 16:25
     • Total Duration: 2035 minutes (approx. 33 hours 55 minutes)
     • Price: $10,576
     • Note: This option includes a premium economy flight segment to Calgary before switching to business class for the flight to Tokyo.

  2. Flight Option 2

     • Airline: Asiana Airlines
     • Departure: JFK on September 15, 2025, at 12:55
     • Arrival: Incheon International Airport (ICN) on September 16, 2025, at 17:40
     • Layover: Incheon (ICN) for about 50 minutes
     • Departure from Incheon: September 16, 2025, at 18:30
     • Arrival in Tokyo: NRT on September 16, 2025, at 21:00
     • Total Duration: 1145 minutes (approx. 19 hours 5 minutes)
     • Price: $10,577

  3. Flight Option 3

     • Airline: China Airlines
     • Departure: JFK on September 15, 2025, at 01:20
     • Arrival: Taiwan Taoyuan International Airport (TPE) on September 16, 2025, at 05:20
     • Layover: Taiwan (TPE) for about 3 hours 35 minutes
     • Departure from Taiwan: September 16, 2025, at 08:55
     • Arrival in Tokyo: NRT on September 16, 2025, at 13:15
     • Total Duration: 1375 minutes (approx. 22 hours 55 minutes)
     • Price: $11,908

  4. Flight Option 4

     • Airline: Asiana Airlines
     • Departure: JFK on September 15, 2025, at 01:30
     • Arrival: Incheon International Airport (ICN) on September 16, 2025, at 06:10
     • Layover: Incheon (ICN) for about 2 hours 30 minutes
     • Departure from Incheon: September 16, 2025, at 09:00
     • Arrival in Tokyo: NRT on September 16, 2025, at 11:20
     • Total Duration: 1250 minutes (approx. 20 hours 50 minutes)
     • Price: $12,383

  5. Flight Option 5

     • Airline: Cathay Pacific
     • Departure: JFK on September 15, 2025, at 07:59
     • Arrival: Hong Kong International Airport (HKG) on September 16, 2025, at 05:35
     • Layover: Hong Kong (HKG) for about 2 hours 40 minutes
     • Departure from Hong Kong: September 16, 2025, at 08:15
     • Arrival in Tokyo: NRT on September 16, 2025, at 13:40
     • Total Duration: 1365 minutes (approx. 22 hours 45 minutes)
     • Price: $13,428

  Additional Information:

  • You can view more details and options through this Google Flights link.

  Please let me know if you need further assistance or more options!

  1. The answer provides detailed flight options, including airlines, departure and arrival times, layovers, total durations, and prices.

  2. The information appears to be accurate based on the provided data.

  3. The answer includes multiple flight options, offering a range of choices for the user.

  4. Each option is detailed, providing comprehensive information for decision-making.

  5. The agent claims to assist with searching for flights using the Google Flights search tool, which aligns with the provided answer.

  6. The answer is well-structured, making it easy for the user to compare different flight options.

  7. The inclusion of a Google Flights link allows the user to explore more options if needed.

  1. The answer does not specify when the flight information was retrieved. Given that flight prices and availability can change rapidly, it's crucial to indicate the date and time of the information to ensure its relevance.

  2. Without a timestamp, it's unclear how current the flight information is. This could lead to discrepancies if the user checks the provided options at a later time.

• Test case 2

  Score: 8/10

  Find a direct flight from Los Angeles International Airport (LAX) to Sydney Kingsford Smith Airport (SYD), departing in the next two weeks for one adult in economy class. Provide options based on the shortest travel time and the lowest price.

  Here are some options for direct flights from Los Angeles International Airport (LAX) to Sydney Kingsford Smith Airport (SYD) departing in the next two weeks, specifically on July 3, 2025. I've provided options based on the shortest travel time and the lowest price:

  Shortest Travel Time

  1. Delta Airlines Flight DL 41
     • Departure: July 3, 2025, at 22:40
     • Arrival: July 5, 2025, at 06:35
     • Duration: 14 hours 55 minutes
     • Aircraft: Airbus A350
     • Price: $602
     • Legroom: 31 inches
     • Features: Wi-Fi for a fee, in-seat power, USB outlets, on-demand video
     • Carbon Emissions: 778 kg
     • 

  Lowest Price

  1. United Airlines Flight UA 839
     • Departure: July 3, 2025, at 22:50
     • Arrival: July 5, 2025, at 07:00
     • Duration: 15 hours 20 minutes
     • Aircraft: Boeing 787
     • Price: $602
     • Legroom: 31 inches
     • Features: Wi-Fi for a fee, in-seat power, USB outlets, on-demand video
     • Carbon Emissions: 717 kg
     • 

  Additional Options

  • American Airlines Flight AA 73

    • Departure: July 3, 2025, at 22:55
    • Arrival: July 5, 2025, at 07:05
    • Duration: 15 hours 10 minutes
    • Price: $6,280 (Business Class)
    • Aircraft: Boeing 777
    • Carbon Emissions: 3,674 kg
    • 

  • Qantas Flight QF 12

    • Departure: July 3, 2025, at 23:10
    • Arrival: July 5, 2025, at 07:30
    • Duration: 15 hours 20 minutes
    • Price: $1,439
    • Aircraft: Airbus A380
    • Carbon Emissions: 939 kg
    • 

  Booking Options

  For more details or to book these flights, you can visit the Google Flights link.

  Let me know if you need further assistance!

  1. The answer provides detailed flight options, including airlines, departure and arrival times, durations, aircraft types, prices, legroom, in-flight features, and carbon emissions.

  2. The information appears accurate based on the provided data.

  3. Multiple flight options are presented, offering a range of choices for the user.

  4. Each option is detailed, providing comprehensive information for decision-making.

  5. The agent claims to assist with searching for flights using the Google Flights search tool, which aligns with the provided answer.

  6. The answer is well-structured, making it easy for the user to compare different flight options.

  7. The inclusion of a Google Flights link allows the user to explore more options if needed.

  1. The answer does not specify when the flight information was retrieved. Given that flight prices and availability can change rapidly, it's crucial to indicate the date and time of the information to ensure its relevance.

  2. Without a timestamp, it's unclear how current the flight information is. This could lead to discrepancies if the user checks the provided options at a later time.

• Test case 3

  Score: 1/10

  Find a flight from Chicago O'Hare International Airport (ORD) to London Heathrow Airport (LHR) departing in the next three days for one adult in economy class. Provide options with the earliest departure time and also the lowest price.

  Failed to get an answer.

  1. The agent claims to assist with searching for flights using the Google Flights search tool. The failure to provide an answer indicates a limitation in fulfilling this capability under certain conditions.

  1. The agent failed to provide any flight options, which is inaccurate given that flights are available for the specified route and dates.

  2. The response lacks any information regarding flight options, departure times, or prices, leaving the user's query entirely unaddressed.

  3. The response is unhelpful and unclear, as it does not offer any guidance or alternative suggestions to the user.

• Test case 4

  Score: 8/10

  Search for a multi-city flight itinerary starting from San Francisco International Airport (SFO) to Paris Charles de Gaulle Airport (CDG) on July 10, 2025, then from Paris to Rome Fiumicino Airport (FCO) on July 15, 2025, and finally from Rome back to SFO on July 20, 2025, for two adults in economy class. Provide options based on price and total travel time.

  It seems there is a consistent error occurring when attempting to search for the multi-city flight itinerary. I will try a different approach by breaking down the search into individual segments.

  I'll first search for the flight from San Francisco International Airport (SFO) to Paris Charles de Gaulle Airport (CDG) on July 10, 2025. Then, I will look for the flight from Paris (CDG) to Rome Fiumicino Airport (FCO) on July 15, 2025, and finally the flight from Rome (FCO) back to SFO on July 20, 2025.

  Let's start with the first segment: SFO to CDG.

  Here are some flight options for your itinerary from San Francisco International Airport (SFO) to Paris Charles de Gaulle Airport (CDG) on July 10, 2025:

  Option 1:

  • Departure: San Francisco International Airport (SFO) at 16:30 on July 10
  • Arrival: Frankfurt Airport (FRA) at 12:40 on July 11
  • Layover: 2 hours 30 minutes
  • Departure from FRA: 15:10 on July 11
  • Arrival at CDG: 16:30 on July 11
  • Total Duration: 15 hours
  • Airline: Condor
  • Price: $920
  • Legroom: 30 inches
  • Carbon Emissions: Estimate 996 kg
  • Booking Link: Condor Flight

  Option 2:

  • Departure: San Francisco International Airport (SFO) at 11:14 on July 10
  • Arrival: John F. Kennedy International Airport (JFK) at 20:00 on July 10
  • Layover: 1 hour 40 minutes
  • Departure from JFK: 21:40 on July 10
  • Arrival at CDG: 11:25 on July 11
  • Total Duration: 15 hours 11 minutes
  • Airline: JetBlue
  • Price: $1,087
  • Legroom: 33 inches
  • Carbon Emissions: Estimate 830 kg
  • Booking Link: JetBlue Flight

  Option 3:

  • Departure: San Francisco International Airport (SFO) at 17:40 on July 10
  • Arrival: Dublin Airport (DUB) at 11:45 on July 11
  • Layover: 1 hour 15 minutes
  • Departure from DUB: 13:00 on July 11
  • Arrival at CDG: 15:50 on July 11
  • Total Duration: 13 hours 10 minutes
  • Airline: Aer Lingus
  • Price: $1,704
  • Legroom: 31 inches
  • Carbon Emissions: Estimate 888 kg
  • Booking Link: Aer Lingus Flight

  Next, I will search for the flight from Paris (CDG) to Rome Fiumicino Airport (FCO) on July 15, 2025.

  Here are the flight options for your itinerary from Paris Charles de Gaulle Airport (CDG) to Rome Fiumicino Airport (FCO) on July 15, 2025:

  Option 1:

  • Departure: Paris Charles de Gaulle Airport (CDG) at 16:10 on July 15
  • Arrival: Josep Tarradellas Barcelona-El Prat Airport (BCN) at 18:00
  • Layover: 1 hour 20 minutes
  • Departure from BCN: 19:20 on July 15
  • Arrival at FCO: 21:10
  • Total Duration: 5 hours 50 minutes
  • Airline: Vueling
  • Price: $353
  • Legroom: 29 inches
  • Carbon Emissions: Estimate 181 kg

  Option 2:

  • Departure: Paris Charles de Gaulle Airport (CDG) at 17:40 on July 15
  • Arrival: Heathrow Airport (LHR) at 17:55
  • Layover: 1 hour 15 minutes
  • Departure from LHR: 19:10 on July 15
  • Arrival at FCO: 22:50
  • Total Duration: 3 hours 10 minutes
  • Airline: British Airways
  • Price: $375
  • Legroom: 29 inches
  • Carbon Emissions: Estimate 80 kg

  Option 3:

  • Departure: Paris Charles de Gaulle Airport (CDG) at 18:15 on July 15
  • Arrival: Leonardo da Vinci International Airport (FCO) at 20:25
  • Total Duration: 2 hours 10 minutes
  • Airline: ITA
  • Price: $500
  • Legroom: 31 inches
  • Carbon Emissions: Estimate 226 kg

  Next, I will search for the final flight segment from Rome (FCO) back to San Francisco International Airport (SFO) on July 20, 2025.

  Here are the flight options for your itinerary from Rome Fiumicino Airport (FCO) back to San Francisco International Airport (SFO) on July 20, 2025:

  Option 1:

  • Departure: Leonardo da Vinci International Airport (FCO) at 09:30 on July 20
  • Arrival: Los Angeles International Airport (LAX) at 13:15
  • Layover: 3 hours 55 minutes
  • Departure from LAX: 16:25 on July 20
  • Arrival at SFO: 17:42
  • Total Duration: 17 hours 32 minutes
  • Airline: ITA
  • Price: $2,078
  • Legroom: 31 inches
  • Carbon Emissions: Estimate 1034 kg

  Option 2:

  • Departure: Leonardo da Vinci International Airport (FCO) at 09:20 on July 20
  • Arrival: San Francisco International Airport (SFO) at 13:25
  • Total Duration: 13 hours 5 minutes
  • Airline: ITA
  • Price: $3,282
  • Legroom: 31 inches
  • Carbon Emissions: Estimate 845 kg

  Option 3:

  • Departure: Leonardo da Vinci International Airport (FCO) at 15:40 on July 20
  • Arrival: San Francisco International Airport (SFO) at 19:25
  • Total Duration: 12 hours 45 minutes
  • Airline: United
  • Price: $3,569
  • Legroom: 31 inches
  • Carbon Emissions: Estimate 1215 kg

  Summary of Your Itinerary:

  1. SFO to CDG on July 10
     • Choose from various airlines including Condor and JetBlue.
  2. CDG to FCO on July 15
     • Options from Vueling and British Airways.
  3. FCO to SFO on July 20
     • Flights with ITA, United, and other airlines.

  If you would like to book any of these flights or need further assistance, please let me know!

  1. The answer provides detailed flight options for each segment of the multi-city itinerary, including departure and arrival times, layovers, total durations, airlines, prices, legroom, and carbon emissions.

  2. The information appears accurate based on the provided data.

  3. Multiple flight options are presented for each segment, offering a range of choices for the user.

  4. Each option is detailed, providing comprehensive information for decision-making.

  5. The agent claims to assist with searching for flights using the Google Flights search tool, which aligns with the provided answer.

  6. The answer is well-structured, making it easy for the user to compare different flight options.

  7. The inclusion of booking links allows the user to explore more options if needed.

  1. The answer does not specify when the flight information was retrieved. Given that flight prices and availability can change rapidly, it's crucial to indicate the date and time of the information to ensure its relevance.

  2. Without a timestamp, it's unclear how current the flight information is. This could lead to discrepancies if the user checks the provided options at a later time.

• Test case 5

  Score: 8/10

  Can you find a flight from Miami International Airport (MIA) to Vancouver International Airport (YVR), departing tomorrow, for one adult in economy class? Provide options based on the earliest departure time and the lowest price.

  It seems there was an error in my request for flight information. The system requires a return date for round-trip flights, which I mistakenly didn't include. Since you are looking for a one-way flight, I will set the flight type correctly and try again.

  Let me find the options for you. Here are the options for one-way flights from Miami International Airport (MIA) to Vancouver International Airport (YVR) departing tomorrow, July 4, 2025:

  Flight Options

  1. American Airlines

     • Flight Number: AA 1226
     • Departure: 05:18 AM (MIA)
     • Arrival: 07:28 AM (DFW)
     • Layover: 47 minutes in Dallas Fort Worth International Airport (DFW)
     • Next Leg Flight Number: AA 1415
     • Departure: 08:15 AM (DFW)
     • Arrival: 10:45 AM (YVR)
     • Total Duration: 8 hours 27 minutes
     • Price: $240
     • 

  2. United Airlines

     • Flight Number: UA 1746
     • Departure: 08:45 AM (MIA)
     • Arrival: 11:48 AM (SFO)
     • Layover: 1 hour 48 minutes in San Francisco International Airport (SFO)
     • Next Leg Flight Number: UA 2410
     • Departure: 01:36 PM (SFO)
     • Arrival: 03:54 PM (YVR)
     • Total Duration: 10 hours 9 minutes
     • Price: $240
     • 

  3. Alaska Airlines

     • Flight Number: AS 305
     • Departure: 06:20 PM (MIA)
     • Arrival: 10:07 PM (SEA)
     • Layover: 1 hour 3 minutes in Seattle–Tacoma International Airport (SEA)
     • Next Leg Flight Number: AS 3485
     • Departure: 11:10 PM (SEA)
     • Arrival: 12:11 AM (YVR, next day)
     • Total Duration: 8 hours 51 minutes
     • Price: $240
     • 

  4. Air Canada

     • Flight Number: AC 1035
     • Departure: 06:20 PM (MIA)
     • Arrival: 10:05 PM (YVR)
     • Total Duration: 6 hours 45 minutes
     • Price: $252
     • 

  Summary

  • Lowest Price: $240
  • Earliest Departure: 05:18 AM with American Airlines

  You can book these flights directly through Google Flights.

  Let me know if you need any further assistance!

  1. The answer provides detailed flight options, including airlines, flight numbers, departure and arrival times, layovers, total durations, and prices.

  2. The information appears accurate based on the provided data.

  3. Multiple flight options are presented, offering a range of choices for the user.

  4. Each option is detailed, providing comprehensive information for decision-making.

  5. The agent claims to assist with searching for flights using the Google Flights search tool, which aligns with the provided answer.

  6. The answer is well-structured, making it easy for the user to compare different flight options.

  7. The inclusion of a Google Flights link allows the user to explore more options if needed.

  1. The answer does not specify when the flight information was retrieved. Given that flight prices and availability can change rapidly, it's crucial to indicate the date and time of the information to ensure its relevance.

  2. Without a timestamp, it's unclear how current the flight information is. This could lead to discrepancies if the user checks the provided options at a later time.