As an attorney who focuses in this area, I can say that the most interesting question is first where this case will be decided. Texas, and the Northern District of Texas in particular, has historically been the worst jurisdiction in the country for web scrapers to litigate. Southwest has a long history of litigating successfully there, including two cases from just last year. If TPG is going to win, first they'll need to win the jurisdictional question of whether the case will be decided in Delaware or Texas.
Years ago I (and a handful of other folks) had a meeting with some people from American who were thinking about opening up their data via an API. One of the other attendees said something to the effect of, "software developers are very, very good at removing inefficiencies when given data like this." It was delicately phrased but the subtext was clearly a warning: if your business depends on asymmetry, an API can sink you. I guess they took that warning to heart.
I wrote a scraper for Air Canada's aeroplan program a few years ago. I wanted to track my points in my own custom native app. I probably had $10,000 worth of points in my account. One day I logged in to find out my account had been deactivated on suspicions of fraud. After several lengthy phone calls with their team (including sending them the node.js script I was using), I was able to get my account restored. For the weeks it took to fix my account it was pretty frustrating. I just don't understand why you can't write a script that acts as your web user agent.
"Partly" reminds me of LinkedIn's case against hiQ
> The LinkedIn dispute arose out of hiQ’s use of automated bots to scrape massive amounts of information from publicly available LinkedIn user profiles. Thus far, lower courts have sided with hiQ on grounds that certain information on the site is publicly available and could be accessed by the public without entering a password. [1]
There are similarities, however, different context in that in hiQ's case, information was publicly available, but in TPG's case the owner of that data (the AA customer) is providing them access to the data. The customer could just as well copy / download / screenshot the data, etc, and transfer it to TPG, obviously most people wouldn't bother, so that should be the core of the argument here. Is a user allowed to make their data available to a third-party? Screen-scraping is a means to an end
[1] https://news.bloomberglaw.com/us-law-week/supreme-court-scra...
Why the headline editorialization? "Is screen scraping illegal" isn't in the article headline and scree-scraping isn't mentioned by name. And this case doesn't appear to be generic screen-scraping of a public site, as TPG was using customer credentials to log-in and retrieve info (with permission). The lawsuit is about breach of site T&C.
A bit of an aside - but does anyone here have experience with insurance policies that would actually cover legal defense in a case like this?
Would Tech E&O cover something like this, or are there riders that would need to be added?
It seems like something that could be strongly defended, and Red Ventures (TPG owner) is a large conglomerate so I doubt legal funds is an issue.
I can't see how American Airlines will prevail in preventing screen scraping. At best, they can prevent their point data from being transmitted and stored to TPG's servers.
Screen Scraping is essentially interacting with the DOM extract information. American Airlines can't conceivably attempt to limit programmatic interaction with the DOM because that is a core component of how the web works including accessibility tools such as screen readers, and browser plugins/extensions.
Gotta love Streisand Effect. I did not know TPG had an app, and now it’s on my Home Screen. Thanks, AA!
If screen scraping is illegal just know that pretty much every budget and banking app is also illegal because they all use screen scraping.
The trademark/copyright stuff I get but I don't see how the screen scraping could be illegal.. How is it any different than a person logging in and writing it down manually..
"Illegal" is the headline is the wrong word here. There are no criminal proceedings, just a civil case. Breach of contract is all that is being alleged here.
I wonder if other services like Mint, Yodelee etc. also do scraping? It seems to be the same model there as TPG/AA - a company has user data, but the user wants the data in some other place, so they authorize a third-party to extract the data and represent it in a different place. Most banks now are begrudgingly coming to terms with this being a thing, some going as far as providing OAuth-like read-only APIs to aggregators. Some are trying hard to not let that happen, and some just ignore the issue and let the aggregators scrape. But an actual case decision in this matter could change the picture - and make financial aggregator business so much harder if it's going a wrong way.
I honestly don’t know who to root for here. TPG is one of my favorite sites, but American is one of my favorite airlines.
The web is broken.
The original promise of client/server services was that the server would provide data on a universal open data format, and the USER AGENT (initially a web browser, but other kinds were expected) would process it in a format to the liking of the user, and satisfying their needs.
Compare this to the current situation where the industry standard is that the servers do indeed provide data through somewhat standardized APIs, but the browser or native app is developed by the same vendor and serves their commercial interests, not those of the user as a customer. The only standard customization recognized to users is light theme / dark theme, and it has only started a few months ago.
It doesn't matter if it is illegal. The process is the punishment in the US.
Unlike every other OECD country, the US does not have the English Rule.
https://en.wikipedia.org/wiki/English_rule_%28attorney%27s_f...
I can't seem to find a reason why "screen scraping" is important here, as oppose to just scraping?
My understanding is that screen scraping is taking a picture of the rendered website and using an OCR or some other sort of recognition tools to extract the data.
If it is just scraping - it should be perfectly legal right?
How is this any different than Plaid? How has Plaid managed to avoid a lawsuit like this?
This question about whether a consumer has a right to enjoy access to information or already paid for services without being advertised to is being tested in businesses everywhere.
Streaming services insert ads for other original content when you hit play.
The grocery store has audio and video advertisements for prepare meals playing on loop you can not avoid if you walk past the meat department.
The problem is a race to the bottom on the pricing consumers perceive, and then recovering that money by squeezing every possible touch point.
People have to have enough options to choose a company that doesn’t have to make these compromises.
See also yesterday's FP, about a court's ruling on HTML post-processing once it reaches the user:
https://news.ycombinator.com/item?id=29989927
Plaid could file a friend-of-the-court brief here, since they have (presumably!) strong legal grounds to assert that they are legally within their rights to scrape bank websites, as they're doing so as an authorized user-agent, and since browsers are just user-agents, etc.
Someone asked "what is their case, exactly?"
American Airlines is alleging 12 legal claims:
1. Breach of Contract 2. Tortious Interference with a Contract 3. Unfair Competition by Misappropriation 4. Trespass 5. Trademark infringement 6. Dilution 7. Dilution under Texas State Law 8. False Designation of Origin 9. Copyright Infringement 10. CFAA 11. Violation of Texas Harmful Access by Computer Act 12. Unjust Enrichment
They do the same thing to Award wallet even though award wallet uses s browser extension to browse the website in your actual browser.
The funny thing is, American sends points data to my email. I wonder if that sort of thing could be leveraged for something similar.
I used to work at what was basically a microservice shop for airlines. The amount of dark shit those apps get away with is amazing.
The airline industry has been through this before https://en.wikipedia.org/wiki/Web_scraping#United_States
I'm surprised that an affiliate marketing company is willing to get in such a fight. Like I would have thought everything they did was blessed and even paid for by airlines and credit cards.
Is this different than what Plaid does with bank logins?
They also went after AwardWallet and told them to shut it down.
Y'all, think of the poor servers! They can't handle the traffic!!! /s
ex-RV employee - it's a powerhouse of data for every significant moment in your life - from health to utilities to mortgage and everything in between. I personally don't see anything wrong on with this specifically.
If it isn't search engines and news aggregators are illegal.
"Airline miles" are like cryptocurrency -- a fake currency used to run scams. They should be regulated as such.
data yearns to be free
> The interest was monetization of customer eyeballs, an American Airlines source shared that they wanted customers checking accounts at AA.com where they could be marketed to.
It seems like so many problems, annoyances and inconvenience in modern society are artificially created/maintained just to enable this disgusting industry. Imagine how more efficient things could be if this cancer was eradicated once and for all.
One thing that has changed in the airline industry that wasn't the case when the points guy started, is that the Airline Frequent flier miles are worth more than the actual airline themselves making the airlines essentially banks.
https://www.aa.com/i18n/customer-service/support/legal-infor...
"Unless otherwise noted, all information, AAdvantage® account information, articles, data, images, passwords, Personal Identification Numbers ("PINs"), screens, text, user names, Web pages, or other materials (collectively "Content") appearing on the Site are the exclusive property of American Airlines Group, Inc., or American Airlines, Inc., or their subsidiaries and affiliates"
"You may not copy, display, distribute, download, license, modify, publish, re-post, reproduce, reuse, sell, transmit, use to create a derivative work, or otherwise use the content of the Site for public or commercial purposes. Nothing on the Site shall be construed to confer any grant or license of any intellectual property rights, whether by estoppel, by implication, or otherwise."
Seems pretty cut-and-dry to me.
Interesting that AA frames it as TPG's app stealing confidential customer data, while it is the customer who set up the app and willingly provided their credentials in the first place.
Will AA be able to find a single customer who has a problem with what TPG does? What is their case then exactly? Would they similarly sue the app if customers were copy pasting data into it, rather than accessing it programmatically?
FYI the title is editorialized. This suit has nothing to do with screen scraping, but just data access in general. Services like these nowadays almost always use private APIs (built for mobile clients and SPAs) rather than parse HTML.