When the Free Speech Coalition sued to block Texas’s age verification law for porn sites, the Supreme Court upheld that law in Free Speech Coalition v. Paxton (2025). When NetChoice sued to block Mississippi’s age verification law for social media, though, Justice Kavanaugh—who was part of the majority for FSC—wrote a brief opinion suggesting that this law was unconstitutional.

Now, states have passed age verification laws for app stores: the App Store Accountability Act. And more court battles are inevitable. Who will win this time? Here, I’ve read quite a few polished policy pieces that feel more like a pep rally for one side; a mock battle is more my style. In a court battle, my role would be a technical expert, so I’ve written a mock expert declaration in opposition to this law.1

1. When NetChoice sued to block Mississippi’s age verification law for social media, the Fifth Circuit remarked, “This case continues our struggle with the interface of law and the rapidly changing universe of technology.”

2. In this case, comparisons to Free Speech Coalition v. Paxton (2025) are inevitable. Such comparisons, though, again bring this Court back to “the interface of law and the rapidly changing universe of technology.” On the technical side, what similarities—or differences—would be found in the factual records for both cases?

3. On the surface, the factual records seem similar. Age verification technology today would be similar to (if not better than) age verification technology at the time of FSC. The factual record is incomplete, however, unless it also considers the requirements of the age verification law.

4. Compare two software features: finding nearby restaurants and finding nearby friends. They seem similar on the surface, but restaurants rarely change their location, while friends frequently change their location. Unlike nearby restaurants, software to find nearby friends has this requirement: it processes over 100,000 location updates per second.2 Likewise, the App Store Accountability Act (“the Act”) imposes materially different requirements, compared to the law in FSC (“H.B. 1181”).

Comparing the Basic Requirements of Both Laws

NOTE: Since there are multiple versions of the App Store Accountability Act, I’ve focused on the requirements that are common to most, if not all, versions of this act.

5. Based on my understanding of H.B. 1181, it contains these requirements:

• Pornographic sites must block users who are under 18.

• Pornographic sites must verify each user’s age category: under 18, or 18 and older.

6. Based on my understanding of the Act, it contains these requirements:

• App stores may allow users who are under 18, but they must obtain parental consent for each app that an underage user wants to download.

• App stores must verify each user’s age category: under 13, 13 to 15, 16 to 17, or 18 and older.

• App stores must verify parental consent for minors.

Under 18 vs. Granular Age Categories

7. Facial age estimation is a modern method of age verification. To explain how it works, a webcam captures a short video (or an image) of one’s face. An algorithm uses this video to estimate the person’s age. Once that estimate is produced, the video can then be deleted, alleviating privacy concerns. This is known as a biometric method of age verification.

8. Suppose that an engineer was designing an age verification system for pornographic sites; the system only needs to verify if a user’s age is under 18, or 18 and older.

9. An engineer may look at using facial age estimation. This solution would produce the correct age category for most users—though a secondary method would be needed for 18- to 20-year-olds. In its whitepaper on facial age estimation, Yoti reports that it can reliably determine that 13- to 17-year-olds are under 21; the accuracy is 99.3%.

10. According to the Age Verification Providers Association’s amicus brief in FSC, “while biometric age verification cannot perfectly identify a user’s age, it effectively waves in the vast majority of users who are well over 18, leaving potential doubts only as to those between 18 and 21.”

11. But suppose that the requirements are changed: the system must now verify if a user’s age is under 13, 13 to 15, 16 to 17, or 18 and older.

12. Facial age estimation is not perfect. If the margin of error is ±1 year, a user estimated to be 15 could actually be 14 or 16; 16 is a different age category. Per Yoti’s whitepaper, their facial age estimation has a mean absolute error of 1.1 years for 13- to 17-year-olds.3 For many minors, the margin of error would cross the boundaries of an age category.

13. In the offline world, people often verify their age with a driver’s license, so an engineer could try that option next. One age category, though, is 13 to 15. Many users in that category do not have a driver’s license (or even a learner’s permit).

Blocking Underage Users vs. Allowing Underage Users with Parental Consent

14. Suppose that a new requirement is added: the system must verify parental consent for users who are under 18. The key challenge here is verifying the parental relationship between an adult and a minor. (Once that relationship is established, verifying parental consent is straightforward.4)

15. Facial age estimation can only verify an age, not a parental relationship. Likewise, a government ID can only verify an identity and an age, not a parental relationship.5

16. A birth certificate, by contrast, does list a parent, but it can only verify who has custody at the time of birth. Custody can change, for example, if the child is adopted—or for minors with divorced parents, minors in foster care, and emancipated minors.

17. In his testimony for Arkansas’s age verification law, Tony Allen of the Age Verification Providers Association observed that the “biggest challenge” is establishing the parental relationship: “It’s easy to say that this person who is giving the consent is, let’s say, in their 40s, versus the person that’s asking for the consent being under 18. But actually establishing that that is a parent or a legal guardian, that’s the challenge with those processes.”

18. Age is a self-contained fact. I understand that a parental relationship, though, is a government-dependent fact; for example, the government decides who gets custody in a divorce, or whether an adult can adopt a minor. When an engineer designs a system that verifies a parental relationship, navigating custody laws goes beyond the means of what an engineer can do.

The Scope and Scale of the Act

19. Verifying age or parental consent is a means to an end. And I understand that the requirements of both laws differ as to their ends (not just their means). The scope of the Act is not limited to pornographic apps.

Assessing the Scope and Scale

20. I understand that this Court is bound by Moody v. NetChoice (2024), which requires a two-step facial analysis: “The first step in the proper facial analysis is to assess the state laws’ scope.” Based on my understanding of the Act, the scope includes (but may not be limited to) every app that is downloaded from Apple and Google’s app stores.

21. The scale here is defined by both volume and variety. In terms of volume, Apple reported in 2022 that its app store had nearly 1.8 million apps. Estimates indicate that Google’s app store also has millions of apps. In terms of variety, Wired published a story in 2010 about an Apple slogan “so catchy that it's endlessly parroted by the media”: “There’s an app for that.” One TV ad featured an app to check snow conditions on the mountain, an app to count the calories in your lunch, and an app to find where you parked your car.

22. Per Moody, “The next order of business is to decide which of the laws’ applications violate the First Amendment, and to measure them against the rest.” In terms of volume, analyzing millions of apps on an individual basis could go beyond this Court’s capabilities.

23. In terms of variety, while many pornographic sites exist, one can make generalizations that apply broadly to all pornographic sites. An app ecosystem where apps can do just about anything, however, is also an ecosystem that resists meaningful generalizations.

24. It may be possible to anecdotally identify apps that are inappropriate for minors. But even assuming arguendo that the Act is constitutional as applied to these apps, the plural of anecdote is not data. Measuring an ecosystem containing millions of different apps—including both constitutional and unconstitutional applications—is a very different task.

Managing Complexity and Scale

25. At 65 MPH, can a car drive 300 miles from St. Louis to Chicago in 4.5 hours? Consider real-world conditions: traffic, red lights, and roads with speed limits under 65 MPH. The answer is no. Even under ideal conditions—no traffic, no red lights, and speed limits of 65 MPH or greater—the drive would take over 4.5 hours (300 miles / 65 MPH ≈ 4.62 hours).

26. Does the Act’s plainly legitimate sweep include social media apps? This Court may have to confront such constitutional questions not just for social media apps, but for many types of apps. Here, this Court could apply Moody’s framework under ideal conditions for the State—assuming arguendo that the Act’s legitimate sweep includes social media apps.

27. Suppose that—even under ideal conditions for the State—the number of unconstitutional applications grows beyond what Moody allows. In that case, this Court need not consider real-world conditions, such as deciding whether the Act is constitutional as applied to social media apps.

28. Still, analyzing this ecosystem can be difficult without concrete examples. Social media—which relies on user-generated content—is one example. This ecosystem also includes apps that rely on curated third-party content, such as the collection of movies on the Netflix app, and apps that rely on first-party content, such as The New York Times app.

29. Apps that rely on user-generated content do not just include social media apps. They also include user-generated encyclopedias, such as Wikipedia’s app, and user-generated reviews, such as Yelp’s app.

30. This ecosystem includes non-violent video games such as Solitaire, Candy Crush, and Angry Birds. It includes Bible apps and other religious apps. And it includes an app to check snow conditions on the mountain, an app to count the calories in your lunch, and an app to find where you parked your car.

Sharing Age Data with Apps: Cybersecurity and Privacy Risks

31. Based on my understanding of the Act, it contains these requirements:

• App stores must share the (verified) age category of a user with apps. For minors, app stores must also share the parental consent status.

• Apps must verify the age category and parental consent for users, using the data provided by app stores.

The “Zero Trust” Mindset

32. A core cybersecurity challenge is the defender’s dilemma: attackers can strike from anywhere using any method. If hackers are trying to penetrate a corporate network, they could exploit a security vulnerability, or they could embed malware in a PDF and email it to any employee. For the attackers to succeed, the defenders only need to fail once.

33. If the corporate network is a castle, attackers often infiltrate this castle. John Reed Stark, former chief of the SEC’s Office of Internet Enforcement, once said, “Cybersecurity is an oxymoron and a data breach is inevitable.” Thus, the industry developed a “zero trust” paradigm: don’t assume it’s safe inside the castle.

34. If the app store is a castle, both Apple and Google have an app review process to ensure the land inside their castle is safe. This process is not perfect. In 2024, a fraudulent app impersonating LastPass, a popular password manager, was discovered in Apple’s app store. In 2025, the Tea Dating Advice app suffered a data breach that exposed users’ selfies and driver’s licenses.

35. The defender’s dilemma also applies to app stores, but Apple and Google must defend a landscape with millions of different apps—a landscape much larger than a corporate network. And inevitably, bad apps often find ways to infiltrate the castle. Thus, the same “zero trust” principle applies: don’t trust an app just because it’s inside the castle walls of the app store.

Best Practices for Sensitive Data

36. For age data, a useful frame of reference is location data; both are sensitive data. The McDonald’s app wants your location to find nearby restaurants, but data brokers (companies that collect personal data and sell it) also want your location. And parents may object if a browser broadcasts their kid’s location to every site they visit. Thus, the tech industry converged on this principle: do not share location data without the user’s permission.

37. For websites, the World Wide Web Consortium (W3C), a standards body, published a geolocation standard that “requires express permission from an end-user before any location data is shared.” For apps, iOS, Apple’s operating system for iPhones, only shares location with an app with the user’s permission. Likewise, Android, Google’s operating system for smartphones, only shares location with an app with the user’s permission.

38. A similar use/abuse duality exists for age data. Apps can use it to enforce age restrictions. However, data brokers also want this data, and predators could use it to target children. In February 2025, Apple announced plans to share the age category of child accounts with apps.6 However, it would do so “if and only if parents decide to allow this information to be shared, and they can also disable sharing if they change their mind.”

39. Compared to Apple’s approach, the Act lacks a key privacy protection: it forces app stores to share age data with apps without a parent’s permission.

40. From a parent’s perspective, an app developer can reasonably be characterized as a stranger. And not every app developer may work for a reputable tech company. For a parent, the decision to let a child use an app is distinct from the decision to share a child’s personal data—including age data—with an app.

41. Age data can also be combined with other data that the app has already collected. For example, when the Act provides age data to apps that use location data, it introduces this risk: it tells strangers which users are kids, and where those kids are located.

Comparing Incentive Structures to Protect Privacy

42. Based on my understanding, the Act also forbids apps from sharing age data with unaffiliated third parties. In other words, the Act forces app stores to share age data with apps without asking a parent for permission, but then tells apps not to misuse this data.7

43. If an app misuses this data, someone must catch them. Here, app stores and law enforcement face a vast, ever-changing landscape with all three Vs of Big Data: volume, variety, and velocity. For volume and variety, millions of different apps exist. For velocity, many app developers regularly update their apps; app stores must review a steady stream of updates.

44. Even if an app is caught, the app developer, for example, could live in China and have ties to the Chinese military.8 In August 2025, security researchers at Comparitech found 10 VPN (virtual private network) apps that communicated with Russian domains and 6 that communicated with Chinese domains. Earlier research from the Tech Transparency Project traced multiple VPN apps back to QiHoo 360, a Chinese military company.

45. In FSC, I understand that the Court concluded that pornographic websites would “have every incentive to assure users of their privacy.” Here, though, it would not be prudent to assume that millions of different apps would all be incentivized to protect privacy—especially in the case of Chinese or Russian apps.

Data Minimization

46. Another privacy principle is to only collect or share data where there is a specific, legitimate purpose for using that data. (Nonetheless, collecting or sharing a large amount of data still presents privacy risks, even with a specific, legitimate purpose.)

47. Here, one such purpose is denying access to an app. But this purpose can be accomplished without sharing age data with apps. If the burden of verifying age and parental consent is shifted to the operating system (OS), the OS can block an app from starting if a parent blocks that app.910

48. Second, apps could build child safety features that depend on an age category. But not all apps may build such features. Even for those that do, Apple or Google may want to first verify that the app has a specific, legitimate purpose to use an age category. The Act, by contrast, forces app stores to unconditionally share an age category with every app.

49. There exists, however, a third purpose. I understand that some child safety regulations are triggered when a site or app has actual knowledge of a user’s age, such as certain provisions in the Children’s Online Privacy Protection Act.11 Such laws can create an incentive structure where sites and apps can reduce regulatory risk by avoiding actual knowledge of their users’ ages. Child safety advocates have often criticized the actual knowledge standard; some even claim it is “almost impossible to prove in a court of law.”

50. If an app verifies a user’s age, it then has actual knowledge of a user’s age.12 This is merely an idea, though. Legislators must still chart out a path to implement this idea.

51. The Act charts out this path. First, it places the burden of age verification on app stores. Then, it forces app stores to share that verified age category with every app. On the surface, this solution appears to work. Nearly every app has actual knowledge of its users’ ages.

52. Having considered how age data is used to create actual knowledge, the next step is to consider how this same data can be abused. Recall that apps should be untrusted by default; “zero trust” applies. In a world filled with hackers, data brokers, and predators, the Act forces app stores to share age data with millions of untrusted apps—without asking a parent for permission. This is a dangerous path.

Concluding Note

53. Justice Thomas—who authored the majority opinion in FSC—asked the first question of oral arguments: “Can age verification systems ever be found constitutional?” I offer no opinions on the constitutionality of age verification as an idea. In the context of a case or controversy, though, I can offer my expertise concerning an implementation of that idea. A different implementation with different requirements would be evaluated differently.

Age verification bills often raise privacy concerns, but does age verification for app stores offer a novel approach that avoids these concerns? In written testimony to Texas’s legislature, Joel Thayer, who “developed the legal and policy framework” for this approach, claimed that the answer is yes:

The Act even avoids the obvious privacy objection that Big Tech organizations like to lodge against age verification measures at the website tier. App stores already have all of this age information. This means that the user would not need to proffer more data to these platforms — a distinct characteristic from website-level age verification requirements.

After the federal version of this bill was introduced, many prominent supporters also made similar claims. It’s a rather remarkable claim about privacy—one that could easily persuade many people to support this act. It’s also a false claim; users would have to “proffer more data to these platforms.”

“App stores already have all of this age information.” Thayer’s mistake here is subtle but important: an inability to distinguish between an attested age and a verified age.

App stores do already have age information, but what they have is an attested age.

Let’s use Google as an example. Google’s app store knows who you are—or perhaps more accurately, who you claim to be—based on your Google Account. And you did tell Google your age when you created that Google account.

But did Google make any attempt to verify that age? No. You can lie about your age here, just like you can lie when a website asks you to enter your age. What Google has collected is merely an attested age, not a verified age. (It’s the same for Apple.)

The App Store Accountability Act, however, requires a verified age. And to verify that age, you will need to provide more personal data to Apple or Google.

To make matters worse, the App Store Accountability Act age-gates every app, not just apps that may be inappropriate for children. If you want to download a Bible app, you will need to verify your age with Apple or Google first.

While I’m here, I want to briefly make another key point: when evaluating the privacy concerns of age verification, the specific requirements of an age verification bill matter just as much, if not more, than the current state of age verification technology.

From a privacy perspective, these are two dramatically different bills:

1. I just need to verify if a person is 18 or older.

2. I need to verify if a person is 12-, 13-15, 16-17, or 18+. For users under 18, I also need to verify parental consent—which requires verifying a parental relationship. Also, I will force app stores to share age data with millions of app developers.

For example, while facial age estimation is a newer and more privacy-conscious method of age verification, it doesn’t work if you need granular age categories, such as 13-15 or 16-17. And that’s just scratching the surface…

Conduct, Not Content

“Does this bill violate the First Amendment?”

When a legislator proposes the App Store Accountability Act—or really any bill to protect kids online—a small army of experts (or “experts”) will often show up at their door, often armed with arguments that their bill violates the First Amendment.

But while the legislator justifiably expresses skepticism towards these hostile experts, the experts then add that the courts have sided with them. NetChoice (a trade association for Big Tech) has sued to block age verification laws for social media in Arkansas, Ohio, Mississippi, and Utah. In all four states, the judge sided with them.

Thus, the legislator begins to question whether their law is unconstitutional. That question is too imprecise for my tastes, though. I would instead ask more precise questions—ones that are akin to what a practicing lawyer would ask.

First, what level of scrutiny applies: rational-basis review, intermediate scrutiny, or strict scrutiny? Second, does the law survive that level of scrutiny?1

And since we’re predicting what courts would do in the real world, let’s examine a real-world case: TikTok v. Garland (2025). Here, the Supreme Court considered a law that forces China to divest TikTok—and bans TikTok if China does not divest.

For many experts, their time to shine had come. On one side, Jennifer Huddleston of the Cato Institute confidently proclaimed that the law was unconstitutional under strict scrutiny. On the other side, Joey Thayer of the Digital Progress Institute confidently proclaimed that there are “no [First Amendment] concerns here,” as “[t]he bill regulates TikTok's conduct, not content.” (For what it’s worth, this engineer predicted that courts would uphold the law under intermediate scrutiny.)

Whose elaborate legal theories would survive contact with reality? When the Supreme Court handed down its unanimous decision, it upheld the law under intermediate scrutiny—a standard used when some First Amendment concerns exist.2

What about the App Store Accountability Act? Friendly experts have declared that this bill regulates conduct, not content—describing it as a bill that regulates contracts with minors, not as a content bill. I can spot the logical fallacy there.

That argument creates a false dichotomy. It suggests that we can cleanly separate laws into conduct laws—which are subject to rational-basis review—and content laws—which are subject to strict scrutiny. In the real world, however, many laws fall somewhere in between a conduct law and a content law.

The true choice courts must make is not between strict scrutiny and rational-basis review, but between strict and intermediate scrutiny. As the Supreme Court said in Moody v. NetChoice (2024), “In the usual First Amendment case, we must decide whether to apply strict or intermediate scrutiny.”

If an age verification law touches social media, it probably qualifies as the “usual” case. The App Store Accountability Act touches social media apps (and other apps).

Of course, some experts—armed with motivated reasoning—will then argue that this law is the “unusual” case. (It’s not.) Ohio already tried to claim that age verification laws are contract laws, not content laws. Judge Marbley rejected that claim: “this Court is unaware of a ‘contract exception’ to the First Amendment.”

And if these experts still won’t back down, I would then ask this: what if this was the usual case? Can this act not even survive intermediate scrutiny?

If the App Store Accountability Act could survive intermediate scrutiny, then perhaps there’s little harm in shooting your shot for rational-basis review. But if the act can’t survive intermediate scrutiny, then your entire legal argument depends on the courts applying rational-basis review. Engineers would call that a single point of failure.

And if that single point of failure collapses if the courts decide that this is the usual case where rational-basis review does not apply, then the act is poorly engineered.

Thus, going forward, we will assume that this is the usual case where “we must decide whether to apply strict or intermediate scrutiny.”

Holding Dreamwidth Accountable

In many cases, age verification bills easily pass the legislature, but fail to survive contact with reality in the courts. Before we jump into the App Store Accountability Act, we should first study those failures in the realm of social media.

After Utah passed its age verification law for social media, NetChoice sued Utah in December 2023—and the Foundation for Individual Rights and Expressions followed suit in January 2024. NetChoice had already convinced Judge Brooks to block a similar law in Arkansas in August 2023. After they sued Utah, NetChoice also convinced Judge Marbley to block a similar law in Ohio in February 2024.

Needless to say, Utah legislators could not ignore the legal threat from NetChoice as they contemplated their next steps. Friendly experts, led by the Institute for Family Studies (IFS), published a coalition letter in February 2024 with this message: “Don’t back down now.” “Utah is retreating at the very moment of Big Tech’s vulnerability.”

Regarding the legal concerns, they wrote, “attempts to forecast what . . . will survive judicial review are highly speculative.” They declared that Utah’s law is “not a restriction on speech (though Big Tech lobbyists have been arguing otherwise),” as it “regulates minors’ right to contract for certain goods and services.”

(This letter was published after Judge Marbley in Ohio had written that “this Court is unaware of a ‘contract exception’ to the First Amendment.”)

Meanwhile, I sought to study the words of Judge Brooks and Judge Marbley. Even if you do not agree with them 100%, you still must respect their authority as judges. And, I suspected that those laws may have been poorly engineered.

Eventually, I zeroed in on one key flaw: a bad definition of social media. The Internet contains over one billion websites; how do you accurately classify each one as “social media”/“not social media”? Speaking from experience, it’s not as easy as it seems.

As I warned in City Journal: “Legislators in both parties may want to hold Big Tech accountable, but their reforms will go for naught unless they sweat the details of their definition.” Specifically, I focused on three common anti-patterns: content-based exceptions, overinclusive definitions, and vague definitions.

In some circles, the reaction to my fixation on the definition was not exactly positive. Was this a pointless “crusade”? In other cases, this fixation was just ignored.

Meanwhile, NetChoice was litigating social media laws in Mississippi and Texas. Both states followed a familiar pattern with their definition. The first part of the definition was overinclusive; it included sites that weren’t social media. Thus, the second part of the definition contained exceptions that would exclude those sites.

NetChoice, however, argued that some exceptions were content-based. Judge Ozerden in Mississippi and Judge Pitman in Texas both agreed, blocking those laws, respectively, in July and August 2024. That same objection—content-based exceptions—had also previously persuaded Judge Brooks and Judge Marbley.

What goes wrong when a definition is “content-based”? Content-based laws are subject to strict scrutiny (while content-neutral laws are only subject to intermediate scrutiny). And strict scrutiny tends to be “strict in theory, fatal in fact.” In short, content-based exceptions tend to be fatal in fact—and were fatal in four states.

To return to Utah, in reaction to NetChoice’s legal threat, legislators decided to stay the course, but they also could not walk the exact same path. And although they acted before the court decisions in Mississippi and Texas came down, they nonetheless sensed that a definition with 20 exceptions would be a vulnerability, so they rewrote it.

This time, NetChoice could not convince the judge that the definition had a content-based exception. They did, however, convince the judge that the definition was overinclusive, noting that it included Dreamwidth (a blogging service). NetChoice won again, and Judge Shelby blocked Utah’s law in September 2024.

The slogan was “hold Big Tech accountable,” not “hold Dreamwidth accountable.” As Judge Shelby noted, “Dreamwidth is distinguishable in form and purpose from the likes of traditional social media platforms—say, Facebook and X.” Dreamwidth was not exactly a guilty party here. Courts will not give you a pass for punishing the innocent just because your legislation also punishes the guilty.

The Full Scope of a Law

Perhaps the time had come to reflect back and carefully study these failures before charting a path forward. The Institute for Family Studies, however, charged forward—this time with new model legislation for the App Store Accountability Act.

As we said earlier, we will assume that this is the usual case where “we must decide whether to apply strict or intermediate scrutiny.” And for the sake of argument, let’s assume that intermediate scrutiny applies.

Does the law survive intermediate scrutiny? Experts on both sides will bring out elaborate legal theories—and both would be willing to litigate them all the way to the Supreme Court. But before both sides take their legal battle that far, I propose that we first examine what the Supreme Court said the last time a battle like that took place.

Let us conduct the analysis that the Supreme Court laid out in Moody v. NetChoice (2024): “The first step in the proper facial analysis is to assess the state laws’ scope.”

In an op-ed, the IFS and others claimed that their legislation merely “requires brick-and-mortar stores to check ID for purchases of age-restricted products like cigarettes and alcohol.” This fatally flawed analogy grossly misstates the scope of their act.

The correct analogy would be checking ID before you can even enter a brick-and-mortar store, such as a Walmart or a 7/11. To use Utah’s legislation as an example, age verification does not kick in when a user downloads certain apps that are harmful to minors, but when the user “creates an account with the app store provider.”3

Do you want to download a Bible app? You must verify your age—and get parental consent if you’re a minor. The Microsoft Word app? You must verify your age—and get parental consent if you’re a minor. The Dreamwidth app? You must verify your age—and get parental consent if you’re a minor.

If defining social media was such a vexing problem, then perhaps some thought that we could avoid this problem by pivoting to app stores. Not so. The purpose of this definition was to control the scope of the law; we could tune the law’s scope by tuning the definition. The pivot to app stores, however, actually broadened the scope.

Returning to Moody, “[t]he next order of business is to decide which of the laws’ applications violate the First Amendment, and to measure them against the rest.” From there, “[t]he question is whether ‘a substantial number of [the law’s] applications are unconstitutional, judged in relation to the statute’s plainly legitimate sweep.’ ”

Here, the IFS focused on the “heartland applications” of this act, such as social media. But as we look at the full scope of the law, we can find a steady stream of examples—such as a Bible app or the Microsoft Word app—where the age verification and parental consent mandates do not even survive intermediate scrutiny.4

Even if the act is constitutional for those “heartland applications,” we can also identify a substantial illegitimate sweep—rendering the act unconstitutional.

If requiring age verification for Dreamwidth rendered a law overinclusive, then imagine how overinclusive a law must be if it requires age verification for a Bible app.

1. Ban social media for kids under 16.

2. Only allow kids under 16 on social media if they have parental consent.

Which choice should you make?

Initially, your gut instinct says the answer is the first option: a straight ban. When you recall the stories about kids and social media that you’ve heard not just from your constituents—but also from your local community—it’s not hard to understand why.

But shouldn’t you give parents a choice here? On top of that, some groups and some experts will inevitably proclaim that “parental responsibility” is the answer, not the government. By those standards, a straight ban is even more out of the question.

“The answer is parental responsibility, not the government.” The more you think about that, the more you realize it’s an inch-deep argument. It’s not hard to imagine a social media influencer tweeting something like that—and it’s probably not wise to raise kids based on the wisdom of a social media influencer.

In the real world, “parental responsibility” effectively means that Big Tech can make as big of a mess as they want, and it’s the “responsibility” of parents to clean it up.

And tech policy experts aren’t supposed to be the ones with inch-deep arguments. They’re supposed to be the ones with deep knowledge, the ones who can clearly articulate the consequences of either choice.

To understand the consequences of this choice, let’s look at it through the eyes of an engineering director at Meta. This director is blissfully ignorant of politics and unaware of the policy debates over age verification—until one day his higher-ups tell him he’s in charge of implementing age verification to comply with a new law.

At first glance, this seems like a challenging yet feasible problem. If nothing else, it’s nowhere near as bad as GDPR compliance. But then one of the director’s best engineers—the type where you can just tell him to do something, and he finds a way to get it done—unexpectedly raises serious alarms about the complexity of the project.

Why the alarm? It’s not about verifying age. It’s about verifying parental consent—specifically, verifying the parental relationship. How do you navigate the custody laws of even one state (much less 50 states)—especially when you consider kids with divorced parents, kids with foster parents, and other complex custody arrangements?

If you just need to verify age, then perhaps you could use facial age estimation—which has minimal privacy concerns—to verify most of your users.1 However, the law says that kids under 16 can join with parental consent. To implement parental consent, you also need to verify the child’s identity, verify the identity of an adult, and then verify that this adult is actually the parent of the child.

Of course, for that scenario to play out in the first place, the legislature would have to pass a law—which has been the easy part in many states—and the courts would have to uphold that law. It’s the latter part where everything often goes off the rails.

Once you pass a law, expect a lawsuit. When Florida passed its law, House Speaker Paul Renner quipped, “[NetChoice] and Big Tech cronies will launch a lawsuit within seconds of HB3 becoming law.”2 (NetChoice is a trade association for Big Tech.)

What are the consequences of your choice in that legal battle? In short, do you want to fight a one-front war or a two-front war against NetChoice? With a straight ban, it’s a one-front war over age verification. With parental consent, you must defend a second front over verifying parental consent—a front that is much harder to defend.3

NetChoice has scored several victories on that second front. In Arkansas, the state got undercut by its own witness, who testified that “the biggest challenge . . . with parental consent is actually establishing the relationship, the parental relationship.”4

In Mississippi as well, NetChoice scored a win on the parental consent front. The judge there also cited Brown v. Entertainment Merchants Association (2011). In that case, the Supreme Court struck down a California law banning the sale of violent video games to kids without parental consent.5

In Brown, the issue of parental consent—and how you would verify it in practice—also became a stumbling block for that law. And that was in the offline world, where the problem is easier to solve. That does not bode well for the online world.

And the lessons to learn from Brown don’t stop at parental consent. The Supreme Court also raised a more fundamental challenge to California’s law:

Although violent video games are not good for kids, they are not a “dangerous, mind-altering material” either. The same cannot be said for social media; there truly is something “qualitatively different” about it. Why is it that Jonathan Haidt let his 9-year-old ride the subway alone in New York City, but he firmly believes that “[s]ocial media is just not appropriate for children” under 16?6

If you truly believe that kids simply don’t belong on social media, then act on those convictions. Propose a straight ban for kids under 16. Wavering here—even if done with noble intentions—can cost you everything in the courts.

If we do not learn from those losses, history will likely repeat itself with KOSMA.

1. The Core Technical Challenge

In the early days of the Internet, Congress made multiple attempts to protect kids online—namely the Communications Decency Act of 1996 and the Child Online Protection Act of 1998—only to see the courts overturn those laws. How do we ensure that history does not repeat itself with attempts to protect kids from social media?

A. “Ideas are Easy. Execution is Everything.”

John Doerr, a well-known venture capitalist and an early investor in Google, once said, “Ideas are easy. Execution is everything.”

Legislation often starts with a belief that social media is harmful for kids. This belief then leads to an idea. Perhaps the idea is that we should require age verification. Perhaps the idea is that—in the case of KOSMA—social media sites should at least ban users when they know that a user is under 13.

And too often, conflicts in the policy world narrowly focus on those grand debates of ideas. Is age verification constitutional? Does banning kids from social media violate the First Amendment? Many policy analysts will jump straight into those grand debates—which rarely touch on finer details like the definition of social media.2

But what if you are more focused on execution? In that case, you would instead jump straight into the court decisions that have blocked these laws to protect kids online.

Why does the definition of social media matter? When we get down to brass tacks, courts look at not just “what” the law does, but also “who” the law applies to. The definition of social media matters because it determines “who” the law applies to.3

And in five cases, before we even get into “what” the law does, the “who”—the definition of social media—is enough to render the law unconstitutional.4

B. I Know It When I See It, but I Can’t Intelligibly Define It

Justice Potter Stewart famously said this about hardcore pornography: “I know it when I see it.” But does that mean he could easily define hardcore pornography?

We often know what a social media site is when we see it, but that doesn’t mean it’s easy to define social media. How do we create a definition that “intelligibly” defines the kinds of sites “to be embraced within that shorthand description”? Five states—Arkansas, Ohio, Mississippi, Texas, and Utah—have tried and failed.5

At its core, defining social media is a vexing technical challenge.6 There are over one billion websites on the Internet (and Google and Apple each have about two million apps in their app stores). A definition has to accurately classify each one of these billion websites as “social media” or “not social media.”

The part that often goes underlooked here is that the definition must exclude sites that are “not social media.” And the volume and variety of such sites can be mind-boggling: Netflix, the comments section of the New York Times, Amazon, (arguably) LinkedIn, Yelp, Wikipedia, Google’s search engine, Substack, and so on.

It’s already hard enough as it is, but on top of that, the First Amendment will impose some limitations on how we can write that definition.

2. Where Did Things Go Wrong?

When crafting a definition of social media, there are many choices we could make, many paths we could take. But many paths lead to dead ends—and to losses in the courts. And while those five losses can provide insights as to which paths we should not take, they don’t always provide insights as to which paths we can take.

A. Taking Exception to Exceptions

In analyzing KOSMA’s definition of social media [Sec. 102(6)], we can split it into two parts: the base definition [Sec. 102(6)(A)] and the 12 exceptions [Sec. 102(6)(B)].

If you’re crafting a definition, perhaps your first draft, like KOSMA’s base definition, will focus on sites that are “a community forum for user-generated content.”

That first draft, however, will tend to include many sites that are “not social media.” For example, Yelp reviews do count as user-generated content.7

If your definition includes Yelp, how do you fix it? Your first instinct may be to add an exception. Arkansas walked down that path, but one exception led to another and eventually to a definition with 13 exceptions. It was an easy win for NetChoice—a trade association for Big Tech that filed all five of these lawsuits.

That does not bode well for KOSMA, whose definition has 12 exceptions. Laws like that tend to be content-based, and content-based laws almost always lose in court.8

What makes a law content-based? A law that favors liberal speech and disfavors conservative speech would be content-based, but more subtle distinctions can also be content-based. In Arkansas Writers' Project v. Ragland (1987), a law that “taxes general interest magazines, but exempts newspapers and religious, professional, trade, and sports journals” was content-based. In Reed v. Town of Gilbert (2015), a law that treated “temporary directional signs” and “political signs” differently was content-based.

When you are familiar with those court decisions from the offline world, these court decisions in the online world will hardly surprise you. In the four court cases after Arkansas’s, the judge cited Reed when analyzing the law’s definition of social media.

It’s not just the number of exceptions, either. Some exceptions will send your definition straight off the cliff. To exclude sites like Yelp, Ohio’s law had an exception for product review websites, which led to another easy win for NetChoice. The judge said this exception was “easy to categorize” as content-based: “For example, a product review website is excepted, but a book or film review website, is presumably not.”

That also does not bode well for KOSMA, which has this exception:

(vii) Business, product, or travel information including user reviews or rankings of such businesses, products, or other travel information.

This exception could also get KOSMA into trouble:

(vi) Content that consists primarily of news, sports, sports coverage, entertainment, or other information or content that is not user-generated but is preselected by the platform and for which any chat, comment, or interactive functionality is incidental, directly related to, or dependent on the provision of the content provided by the platform.

This exception is almost identical to exceptions in Mississippi’s and Texas’s laws. Again, NetChoice won in both states, convincing a judge that the exception was content-based. The judge in Texas, for example, highlighted how this exception “singles out specific subject matter for differential treatment.”

And while no court decision directly addresses an exception like this, by now, we can safely infer that this exception in KOSMA would probably be content-based:

(iii) Crowd-sourced reference guides such as encyclopedias and dictionaries.

B. “Not Social Media”

How do we get back on the right path? All these content-based exceptions have something in common: they are all for sites that have user-generated content—be it comments, product reviews, or wikis—but that are not social media sites.

The real problem is not content-based exceptions. The real problem goes back to our core technical challenge: classifying over one billion sites as “social media” or “not social media.” The base definition inaccurately classifies many sites with user-generated content as “social media” when they are not social media sites.9

That’s why we can’t just solve the problem by removing those exceptions. To return to our example of Yelp, if we remove all 12 exceptions from KOSMA’s definition of social media, then the definition says that Yelp is a social media site.

If Yelp is a social media site, that just creates a different First Amendment problem: overinclusivity. Consider the path Utah took. They saw what was happening in other states with their exceptions, and they clamped down on exceptions in their law. But NetChoice still won on different grounds: overinclusivity.10 The judge, in particular, criticized the definition because it included Dreamwidth (a blogging service).

The existence of user-generated content is necessary to define social media, but it is not sufficient. The real problem is that KOSMA’s base definition is incomplete. There’s a missing piece (or pieces), and we need to figure out what it is.

C. Void for Vagueness

What happens if a company reads the definition, and it has no idea whether the definition applies to its site or not? In that case, we can run into another constitutional problem: the law is void for vagueness.

What sorts of wrong turns can we make here? Phrases like “primary purpose” or “primarily functions” can give the courts fits; “primary purpose” was too vague in Arkansas, and “primarily functions” was too vague in Mississippi.

While this question may invoke images of lawyers debating arcane and complex language details, a more accurate image would invoke Socrates. Perhaps a think-tank scholar approaches a modern-day Socrates, confident in his ability to discern the “primary purpose” of a site. But as Socrates asks one probing question after another, the man eventually walks away, defeated and uncertain of his abilities.

In Arkansas, it only took one probing question: what is the “primary purpose” of Snapchat? The state’s own witnesses could not agree on the answer—and on whether their law applied to Snapchat. Needless to say, that law was void for vagueness.

The base definition of KOSMA, for its part, does use the phrase “primary function”:11

(iv) as its primary function provides a community forum for user-generated content…

As for the phrase “user-generated content,” here is another probing question: what is the difference between third-party content and user-generated content?

Take Netflix, for example. Netflix did not make most of the movies on its platform; those movies are almost always third-party content. But would these movies count as user-generated content? Colloquially, the answer would probably be no.

Legally, though, how do we draw a line between third-party content that is user-generated and third-party content that is not user-generated? If a site relies on third-party content, how do they know which side of the line they’re on?

While vagueness is a Fifth Amendment concern, the bar is raised when the First Amendment gets involved. In FCC v. Fox Television Stations (2012), the Supreme Court said this requirement is more rigorous when speech is involved.12 This is another reason why you should not be overconfident in your ability to clear that bar.

3. “Special Characteristics” of the Missing Pieces

In sports, refs make mistakes, but when you’re 0-5, it’s probably not the refs’ fault. Have we created a definition that is not overinclusive, content-based, and/or vague? We’re 0-5. Instead of complaining about the refs(/judges)—which we can’t control—let’s alter the definition—which we do control. Let’s build a definition so accurate, so obviously content-neutral, so crystal-clear that it’s an easy call for any judge.

We’ve talked a lot about court losses, but let’s talk about a major court win: the divest-or-ban law for TikTok. TikTok argued that this law was content-based, but the Supreme Court rejected that claim, reaffirming a key principle of Turner Broadcasting System v. FCC (1994): a law is content-neutral “when the differential treatment is ‘justified by some special characteristic of’ the particular medium being regulated.”

We know that social media is harmful for kids, but to craft a definition, we need to know why social media is harmful for kids. What are the “special characteristics” of social media, and why do these characteristics make it harmful for kids? Specifics are needed; we can’t make a hand-wavy claim that it’s the social nature of social media.13

This could be one such characteristic: content moderation at scale is hard. Whenever conservatives complained about Big Tech censorship, a certain class of pundits would retort that “content moderation at scale is hard.” As an engineer, I would concede there is some truth to that, but the principle works both ways.

If content moderation at scale is hard, then is social media really safe for kids? Should we be skeptical of the claim that better content moderation will magically solve our problems? It’s not just the engineer saying that; Jonathan Haidt raised a similar point:

How do we execute this idea that content moderation at scale is hard? Add a threshold for daily active users to our definition: one million daily active users who create content.14 (The exact number is negotiable; I had to pick some number here.15)

We can also resolve both of our vagueness concerns. There’s no need to discern the “primary function” of a site with that many active users; just delete that part. And even if movies count as user-generated content, Netflix comes nowhere close to having one million (or even ten thousand) daily active users who create content.

Our odds of winning against NetChoice just got a whole lot better.16

Nonetheless, to return to our core technical challenge—classifying over one billion sites (and millions of apps) as “social media” or “not social media”—the definition could still be overinclusive, even with this new piece.17 There could still be other ways where it classifies a site as “social media” when it is not a social media site.18

But each time that happens, the same pattern applies to find another piece. We find a special characteristic of social media, explain why that characteristic makes social media harmful for kids, and incorporate that characteristic into the definition.

With each additional piece, the odds of winning against NetChoice increases.

In the annals of bad tech bills, few are as infamous as the Stop Online Piracy Act of 2011. Then-Rep. Jason Chaffetz famously said, “We’re going to do surgery on the Internet, and we haven’t had a doctor in the room tell us how we’re going to change the organs. We’re basically going to reconfigure the Internet and how it’s going to work without bringing in the nerds.”

Apps are as ubiquitous today as the Internet was back in 2011, and the App Store Accountability Act—an age verification bill for app stores—would perform surgery on not just app stores, but the entire app ecosystem. But who are the surgeons here, and do they know what they’re doing?

As part of its Family First Technology Initiative, the Institute for Family Studies (IFS) has proposed model legislation for the App Store Accountability Act. At the end of last year, bills based on this model legislation were introduced in both the House of Representatives and the Senate. Recently, South Carolina and Utah introduced bills based on this model.

But will the IFS’s proposed surgery work? Frankly, this engineer—one who has supported age verification for both social media and adult sites—would describe it as medical malpractice.

The Adversarial Mindset

Anyone can create an app—both good actors and bad actors alike. The App Store Accountability would force app stores to share data about your child age’s with all apps—including the bad actors.

When the nerds build products, they have to apply the adversarial mindset. They have to consider not just how regular users will use their product, but also how hackers will try to abuse this product. Security is a de facto requirement for software—unless you want to get hacked.

At its core, cybersecurity is about that never-ending war between attackers and defenders. And it would be fair to say that the Internet can be a very dark place—especially in the corners of the dark web where many hackers reside.

If you’re going to do surgery on the entire app ecosystem, you cannot ignore this war. You, too, must apply the adversarial mindset and consider how attackers might exploit the changes that your legislation will mandate. Security is a de facto requirement for your legislation. (And even if you add privacy protections to your law, it suffices to say that bad actors often don’t obey the law.)

Imagine that bad actors created an app that impersonated Pokémon Go. (Last year, an app impersonating LastPass, a popular password management tool, found its way into Apple’s app store.) As a parent, you give your kids permission to use this app—as you think it is the real thing, not an impostor—giving this app access to real-time location. And thanks to the App Store Accountability Act, app stores are forced to share the age category (e.g., 13-15) of every user with this app.1

In essence, we’ve given predators a database containing the real-time location of kids. And that’s only one of many examples of what could possibly go wrong. A more conventional concern would be data brokers—who don’t always act above-board—acquiring this age data and combining it will all the other information they’ve collected about your kid.

What is stopping the attackers here? The only barrier is the app review process that occurs before an app is added to the app store. And if you believe that Apple and Google cannot be trusted to protect our kids, it does seem odd that you would trust their app review process—and would assume that this process will stop the bad actors who are trying to get their apps into the app store.

But regardless of how you view Apple and Google, this mandate to force app stores to share age data with apps is extremely unwise from a cybersecurity perspective.

Zero Trust

What level of trust should we grant to apps in the app store? From a cybersecurity perspective, experience has shown that you should default to zero trust. In the workplace context, the zero trust mindset assumes that attackers will find a way to breach your corporate network—no matter how good the defenders on your IT team are at securing it—and designs cybersecurity with that in mind. (The zero-trust mindset is not meant as an indictment of your IT team.)

When it comes to app stores, that same zero-trust default should apply. It would be reasonable to assume that—no matter how good Apple and Google’s app review process is—bad actors will find their way inside their app stores. (This is not meant as an indictment of Apple or Google.) And in a zero-trust environment, does it make sense to share age data with untrusted apps? Absolutely not.

The key architects of this model legislation, however, seem to think that it is secure because it uses encryption. In an op-ed, they wrote, “The app store could then transmit the age of the minor user to apps upon download via an anonymous, encrypted signal that indicates whether the user is age-eligible for their product, or not.” Here, it would help to explain what encryption can and cannot do.

If two parties that trust each other are communicating with each other, encryption can ensure that attackers cannot eavesdrop and intercept their communications. However, if legislation is forcing one party to send information to an untrusted party—such as forcing app stores to share age data with an untrusted app—encryption won’t solve that problem.

The Defender’s Dilemma

Another dilemma that we face here is the defender’s dilemma. Defenders tend to have normal(ish) working hours, a spouse and kids, a social life outside of work, etc. Attackers tend to have way too much free time on their hands (and some may live in their mother’s basement), but they are damn good at hacking. Defenders often have to defend a large and complex landscape, and they have to be right 100% of the time. Attackers can attack from anywhere on that landscape, and they only need to be right once. Attackers often outnumber defenders.

For app stores in particular, they have to defend a very large and complex landscape—one with all three Vs of Big Data: volume, velocity, and variety. Both Google and Apple each have about 2 million apps in their app store (volume). They have to review not just the original app, but a constant stream of app updates (velocity). And the apps that they review can be for just about anything (variety).

With all that in mind, how does the app review process manage to keep bad actors out of the app store? The short story is that it doesn’t always work, and it’s not hard to find a treasure trove of stories where it doesn’t work. I say this not to dunk on Apple and Google—despite my misgivings about them—as this would be a very challenging problem for any tech company to solve.

You can certainly understand why the core assumption of “zero trust” makes sense: that some attackers will find their way inside app stores—no matter how good the app review process is.

Minimizing Surface Area

One consequence of the defender’s dilemma is that surface area matters. How large of a surface area are you asking defenders to protect? It’s clear that the IFS did not think enough about that.

The earlier example I gave—impersonating Pokémon Go—is only one of many possible paths. To combine real-time location data with age data, you don’t need to impersonate Pokémon Go; any app that relies on real-time location data is a potential target. Or, instead of impersonating an app, you could also build a Trojan Horse app that act that looks and behaves like a normal app on the outside, but whose real purpose is to help predators.

And that’s not the only way this age data could be misused. As we mentioned earlier, a more conventional concern would be data brokers acquiring this data. A recommendation algorithm run by bad actors could use that age data to help pair kids with predators.

That’s already a very large surface area, and these are mostly examples I came up with off the top of my head. Imagine what bad actors—who have way more time than me—could come up with. Trying to poke a few holes in some of these examples is a fool’s errand, as the hackers—who are smarter than you and also me—could certainly figure out ways to patch those holes, and could also come up with additional examples.

Simply put, forcing app stores to share age data with apps is indefensible—in more ways than one.

0-5. If your football team went 0-5 (be it a professional or college team), it would be time to shake things up. State legislatures who have passed age verification laws for social media have gone 0-5 in legal challenges. Fortunately, this engineer—who has also served as a tech fellow in Congress—has already been figuring out how to fix it.

In part I, we developed model findings that explained the special characteristics of social media. In part II, we’ll dive into the brass tacks of defining social media.

I. The Problem: One Billion Websites

There are over a billion websites on the Internet, and that does not even include all the apps that have online functionality. For social media (or for any online medium) how do we accurately determine which sites are social media sites and which are not?

If you draft a definition with constitutional defects, a lawyer can certainly spot those defects. After you spot those defects, you then need to fix them, which leads back to the core technical challenge: how do you properly classify one billion websites?

A. Technical Challenges

On the surface, the obvious challenge is volume; one billion is a very big number. Beneath the surface, an even bigger challenge exists: variety. Many different types of websites exist, ranging from Facebook to Wikipedia to Netflix.

In the past, attempts to regulate technology were simpler, because the mediums of old were tightly coupled with hardware. Broadcast was defined by an over-the-air signal that an antenna would receive. Cable TV was defined by, well, cable; a cable channel would use “a portion of the electromagnetic frequency spectrum” (see 47 U.S.C. 522(4)).

Online mediums, however, are tied to software—which is much more malleable. A social media site, an app store, and an e-commerce site are all examples of software, but each piece of software yields a radically different medium. The variety of sites available is due in large part to the softness of software.

When writing a definition, including sites that are social media sites is important, but excluding sites that are not social media sites is just as important if not more important. If we are writing an age verification law, we do not want to make people verify their age if they want to leave a review on Yelp.

Yelp is a “negative example,” an example of a site that is not a social media site. Negative examples will play a critical role in developing the definition. As we develop that definition, this phrase will frequently be used: “necessary but not sufficient.”

We will start with a base definition that includes all one billion websites, and we will then incrementally add pieces to that definition. When we add a new piece, this piece will often be motivated by a negative example that the definition needs to exclude. But even with this piece, we can still find other negative examples that are included in the definition. Thus, while this piece is necessary, it’s not sufficient by itself.

B. Legal Challenges

Laws regulating social media will face heightened scrutiny, but the level of scrutiny depends on whether the law is content-based or content-neutral. To make that determination, courts will look at both what the law does, and who the law applies to.

The definition of social media matters because it determines who the law applies to.

Thus far, state legislatures have gone 0-5 when it comes to writing a content-neutral definition of social media. The most common pitfall here is the exceptions.

If you write a definition with thirteen exceptions like Arkansas did, NetChoice will have an easy time convincing the judge that your definition is content-based. Additionally, if a law has thirteen exceptions, it gives lawyers thirteen chances to shoot a law down by proving that one of those exceptions is content-based.

But in some cases, one well-placed shot is enough to bring a law down. In Mississippi, the judge ruled that an exception for “news, sports, commerce, [or] online video games” was content-based. In Ohio, the judge similarly ruled, “The exceptions to the Act for product review websites and ‘widely recognized’ media outlets, however, are easy to categorize as content based.”

Laws are not content-based, however, if they target one medium but not another. Regulations can be justified by the special characteristics of a medium (see Turner Broadcasting System v. FCC (1994)). The Internet is not a monolithic medium; it contains many distinct mediums, such as social media, search, and e-commerce.

But while a definition cannot use a content-based exception to exclude Yelp, if the definition includes Yelp, that creates a different constitutional problem: the definition is not narrowly tailored.

In Utah, the judge criticized their definition of social media because it included Dreamwidth, which is “distinguishable in form and purpose from the likes of traditional social media platforms.” (Dreamwidth is a blogging service that is similar to WordPress, Tumblr, and Medium.)

Finally, what happens if Snapshot cannot determine if it is a social media site, according to the definition? That creates another constitutional problem: vagueness.

Sometimes, this can get technical; for example, multiple judges (though not every judge) have said that phrases such as “primary purpose” are too vague. When Arkansas’s own witnesses could not agree on what the “primary purpose” of Snapchat is, however, that effectively settled that case; their law was void for vagueness.

C. Context Matters

Before we dive into the brass tacks, there is one last important point: the definition may depend in part on the problem we’re trying to solve and the proposed solution.

As a practical example, why can’t we just use the definition in 42 U.S.C. 1862w(a)(2)?

(5) SOCIAL MEDIA PLATFORM.—The term “social media platform” means a website or internet medium that—

(A) permits a person to become a registered user, establish an account, or create a profile for the purpose of allowing users to create, share, and view user-generated content through such an account or profile;

(B) enables 1 or more users to generate content that can be viewed by other users of the medium; and

(C) primarily serves as a medium for users to interact with content generated by other users of the medium.

Taking a quick look, many negative examples will be classified as a social media platform under this definition—which means the definition is not narrowly tailored. (Additionally, the term “primarily” might create some issues in terms of vagueness.)

So why haven’t courts struck down this definition? Look at the rest of 42 U.S.C. 1862w. This definition is used in a law that studies social media’s impact on human trafficking. But what if we use this same definition in a law that regulates social media—as opposed to a law that only studies social media? It probably won’t end well.

A couple more examples will further illustrate this point. First, Florida passed an anti-censorship law for social media (SB 7072, 2021), and it also passed an age verification law for social media (HB 3, 2024). If you compare these two laws, you will find some major differences in how each law defines “social media platform.” Since these two laws are solving two very different problems, though, it’s not surprising that their definitions of “social media platform” would be different as well.

Second, let’s look at a popular federal bill, the Kids Online Safety Act (KOSA). KOSA’s definition of “covered platform” is fairly broad and is not limited to social media. A broader definition makes sense, however, when you look at the solution KOSA is proposing. Since KOSA relies on more light-touch (yet effective) regulations, it does make sense to apply those regulations to a broader set of sites, not just social media.

So what’s the context for the definition that we’re about to create? This definition will be used for child safety legislation. Specifically, it will be used for age verification—though this definition can also be reused for other types of child safety legislation.

For an age verification bill, we will obviously need to write a pretty tight definition of social media. The impact of misclassifying Yelp as a social media site is much more severe for a law that requires age verification (as opposed to extra paperwork). However, if our definition is tight enough that it can be used in an age verification bill, this definition can probably be reused for other child safety bills as well.

II. Classifying One Billion Websites

A. Starting Point

The law is not a creative writing discipline. The goal is to clearly explain to people what the law demands of them—demands that often come with severe penalties if they are violated. In this domain, copying the work of others is good!1

If an existing legal term has a well-understood meaning, or if another law or bill has a well-built definition, just reuse it. Don’t reinvent the wheel. The question is not about you—and whether your legislative work is original. The question is about the people who must obey this law—and whether they understand what the law demands of them.

Where do we start with a definition? Let’s start by reusing the definition of “interactive computer service” from a federal law known as Section 230—the law that says platforms are generally not liable for the third-party content they host.

SOCIAL MEDIA PLATFORM.—The term “social media platform” means an interactive computer service that…

INTERACTIVE COMPUTER SERVICE.—The term “interactive computer service” has the meaning given the term in section 230(f)(2) of the Communications Act of 1934 (47 U.S.C. 230(f)(2)).

For reference, here is Section 230’s definition of “interactive computer service”:

INTERACTIVE COMPUTER SERVICE.—The term “interactive computer service” means any information service, system, or access software provider that provides or enables computer access by multiple users to a computer server, including specifically a service or system that provides access to the Internet and such systems operated or services offered by libraries or educational institutions.

Despite the age of Section 230, which was passed in 1996, this definition of “interactive computer service” is still widely used today. If it’s not broken, don’t fix it.

It also has a modern advantage: it covers both websites and apps. In this definition, it does not matter whether users access Facebook via facebook.com or via the Facebook app; Facebook is an “interactive computer service” either way. (For the sake of convenience, though, we’ll use website/site as a shorthand term for “interactive computer service”—which includes both websites and apps.)

Of course, all one billion websites would qualify as an interactive computer service, so we’ll need to narrow this definition.

B. Content Moderation at Scale is Hard

Section 230 makes a distinction between first-party content and third-party content. When defining social media, that distinction is necessary but not sufficient. Facebook heavily relies on third-party content, but so does Netflix.

1. Big Data and the Three Vs

Back in 2014, Facebook reported that it generates 4 petabytes of data per day (4 petabytes = 4,000,000 gigabytes). By comparison, a typical smartphone has 64 gigabytes of storage, and one of the largest hard drives on the market offers 30 terabytes of storage (30 terabytes = 30,000 gigabytes).

It suffices to say that 4 petabytes of data won’t fit onto a single computer.

Welcome to the world of Big Data. Big Data is defined by the three Vs: volume, velocity, and variety. For example, 4 petabytes is certainly a very large volume of data, and at 4 petabytes per day, the velocity is approximately 46 gigabytes per second.

The three Vs don’t just present technical challenges, either. They also present social challenges. When millions of users are generating content each day, that definitely makes content moderation hard, in terms of both volume and velocity. And in terms of variety, content can cover virtually any topic, and a global social media platform will have content in many different languages.

2. Daily Active Content Providers

So how do we distinguish between Netflix and Facebook? Netflix doesn’t have millions of users who produce content each day. Facebook does. Simply put, scale is the differentiator. After all, our narrative is that content moderation at scale is hard.

In the tech industry, many companies measure their daily active users and monthly active users. We can use this metric as a starting point, but we’ll need to refine it.

First, should we measure daily active users or monthly active users? The answer is daily active users. Velocity—one of our three Vs—is much easier to see at the daily level. Additionally, monthly active users does not distinguish between a user who spends hours on Twitter/X every day and a user who logs in to Twitter/X once a week or once a month; both count as one monthly active user.

Second, we need to tweak what we measure: daily active content providers, not daily active users. If only ten users are producing content but millions of users are viewing that content, content moderation is fairly simple. Both Netflix and Facebook have millions of users, but only Facebook has millions of content providers.

3. Legislative Text

So how do we translate that to legislative text? Fortunately, we have a couple of definitions we can reuse here: the definition of “information content provider” from Section 230, and the definition of “user” from the Kids Online Safety Act (KOSA).

SOCIAL MEDIA PLATFORM.—The term “social media platform” means an interactive computer service that has averaged at least 1,000,000 daily active content providers over the previous 180 days.

DAILY ACTIVE CONTENT PROVIDERS.—The term “daily active content providers” means the number of users who serve as an information content providers during a single day.

USER.—The term “user” means, with respect to a social media platform, an individual who registers an account or creates a profile on the social media platform.

INFORMATION CONTENT PROVIDER.—The term “information content provider” has the meaning given the term in section 230(f)(3) of the Communications Act of 1934 (47 U.S.C. 230(f)(3)).

For reference, here is Section 230’s definition of “information content provider”:

INFORMATION CONTENT PROVIDER.—The term “information content provider” means any person or entity that is responsible, in whole or in part, for the creation or development of information provided through the Internet or any other interactive computer service.

Setting the threshold for daily active content providers is more of an art than an exact science; 100,000 daily active content providers would also be a reasonable threshold.2

C. Scale is Necessary But Not Sufficient

Setting the threshold at 1,000,000 (or even 100,000) daily active content providers will filter out most of those one billion websites, but it’s not a silver bullet. We can still find many negative examples that are included in the definition. Going forward, we will pessimistically assume that scale is necessary but not sufficient.

1. Primary vs. Secondary Content

What about the comments section of the New York Times? The number of users who comment on New York Times articles is much larger than the number of writers who create those articles. And while the New York Times may not have 100,000 or 1,000,000 users commenting each day, perhaps a different site could hit that threshold—which is why we pessimistically assume that scale is necessary but not sufficient. This example will justify that pessimism: what about product reviews on Amazon?

A common mistake here is to create a narrow exception for product review sites. Courts will often rule that that these narrow exceptions are content-based when, to quote an Ohio judge as an example, “a product review website is excepted, but a book or film review website, is presumably not.”

Instead, we can distinguish between “primary” content and “secondary” content. The New York Times article or the product on Amazon would be the primary content, while a comment on the article or a product review would be the secondary content. Secondary content depends on primary content; you cannot add a product review to Amazon if you don’t first have a product to review.

With that in mind, let’s refine the definition of “information content provider”:

INFORMATION CONTENT PROVIDER.—

(A) IN GENERAL.—The term “information content provider” has the meaning given the term in section 230(f)(3) of the Communications Act of 1934 (47 U.S.C. 230(f)(3)).

(B) SECONDARY CONTENT EXCLUDED.—The term “information content provider,” with respect to a social media platform, does not apply to content that depends on other content on the social media platform, such as comments on an article, reviews for a product, or replies to a post.

This exception is written in a content-neutral fashion; product reviews are treated no differently than book reviews or film reviews.3 (A key point here is that the “such as” clause only provides illustrative examples, not an exhaustive list of examples.4 )

Additionally, we can further justify this exception with one of the three Vs: variety. When user-generated content only consists of comments for a small set of articles or reviews for products, the variety of content found on a site is much smaller.

2. Commercial Content

But even if we exclude Amazon reviews, how many people are creating or editing product listings on Amazon each day? How many users create auctions on eBay each day? Again, we pessimistically assume that scale is necessary but not sufficient.

While sites like Amazon and eBay do rely on third-party content, the design of these sites effectively ensures that this content is commercial in nature. We can capture that idea in another exception for “information content provider”:

INFORMATION CONTENT PROVIDER.—

(A) IN GENERAL.—The term “information content provider” has the meaning given the term in section 230(f)(3) of the Communications Act of 1934 (47 U.S.C. 230(f)(3)).

(B) SECONDARY CONTENT EXCLUDED.—The term “information content provider,” with respect to an interactive computer service, does not apply to content that depends on other content on the interactive computer service, such as comments on an article, reviews for a product, or replies to a post.

(C) COMMERCIAL CONTENT EXCLUDED.—The term “information content provider,” with respect to an interactive computer service, does not apply to content that is designed by the interactive computer service to facilitate commerce, such as product listings, available drivers for ridesharing, or booking information for accommodations.

This exception is content-neutral because it makes a medium-based distinction; social media and e-commerce are two fundamentally different mediums.5

For example, if we have a high-volume third-party seller that’s engaging in fraud, verifying the age of the seller would do little to stop that fraud. The regulations created by the INFORM Consumers Act are the answer here. Conversely, it’s also nonsensical to apply INFORM to social media, making social media platforms collect the bank account information, contact information, and tax ID numbers of their users.

3. Directed to a General Audience

Another interesting negative example is LinkedIn. Despite the similarities between LinkedIn and social media, LinkedIn is still intuitively different in form and purpose from social media sites. We can reasonably assume that most parents would not be too concerned if they discovered that their kid secretly created a LinkedIn account.

As before, we need to resist the temptation to exclude LinkedIn by creating a narrow exception for, e.g., professional networking sites. LinkedIn may also not be the only site we need to worry about that has a more specialized purpose; we need an exception that treats all these specialized sites the same:

SOCIAL MEDIA PLATFORM.—The term “social media platform” means an interactive computer service that—

(A) is directed to a general audience, notwithstanding whether content is delivered via text, images, audio, video, or other types of media content; and

(B) has averaged at least 1,000,000 daily active content providers over the previous 180 days.

In terms of content neutrality, this exception is similar to our exception for secondary content. First, it treats all specialized sites the same; it doesn’t privilege certain types of specialized sites.6 Second, it can be justified by one of our three Vs: variety. It suffices to say that the variety of content found on Instagram or Snapchat is vastly bigger than the variety of content found on LinkedIn.

D. The Distribution Model Matters

What about a crowdsourced encyclopedia like Wikipedia? What about email or text messaging? Applying our pessimistic assumption that scale is necessary but not sufficient, the current iteration of our definition still includes these sites.

1. A Variety of Distribution Models

Previously, we focused on how content is created. What if we focus on how content is distributed? It suffices to say that social media’s content distribution model is very different from the distribution model of Wikipedia or text messaging.

Distinguishing between sites based on their distribution model tends to be content-neutral as well. It does not discriminate based on what type of content a site contains, and it enters the territory of a medium-based distinction. Different mediums will have different distribution models; each model presents its own unique set of problems.

Thus, the third plank of our definition will focus on the distribution model:

SOCIAL MEDIA PLATFORM.—The term “social media platform” means an interactive computer service that—

(A) is directed to a general audience, notwithstanding whether content is delivered via text, images, audio, video, or other types of media content;

(B) has averaged at least 1,000,000 daily active content providers over the previous 180 days; and

(C) socially distributes content.

SOCIALLY DISTRIBUTES CONTENT.—The term “socially distributes content” means…

The challenge here lies in defining “socially distributes content.”

2. The Social Network

What is the social component of socially distributing content? As a starting point, Facebook users your social network to distribute content to you; Wikipedia does not. Let’s incorporate the concept of a social network into our definition:

SOCIALLY DISTRIBUTES CONTENT.—The term “socially distributes content” means making decisions about which content to distribute to a user, where such decisions use the user’s social relations with other users, such as other users that the user follows or is friends with.

This definition focuses on decision-making: is this data used to make a decision? Using your social network for other purposes such as research is not covered by that definition; the social network must be used in decisions about content distribution. (Of course, social media sites may take other factors into account when making these decisions, so we only require that the social network be one of the factors used.)

This definition is also crystal-clear; it doesn’t require the courts to make judgments about what the “primary purpose” or “primary function” of a site is.

3. Who Controls Content Distribution?

Social media, however, is not the only social network. If one user subscribes to another user’s newsletter, that’s a social relation that’s used in content distribution; newsletters are distributed to subscribers. If many users are all part of a group chat, that’s a social relation that’s used in content distribution. (Recall that the “such as” clause only provides illustrative examples, not an exhaustive list of examples.)

The next aspect we can look at is who controls distribution. With many of the online mediums that predate social media, consumers and producers (mostly) controlled content distribution. If you send an email, you control who the recipients of that email are. If you don’t like a particular Substack newsletter, you can just unsubscribe.

Of course, this direct distribution model does have one problem: spam. But if spam was the worst of our problems on social media, states wouldn’t be trying to pass age verification laws, and Congress wouldn’t be trying to pass the Kids Online Safety Act.

Search engines typically put users in control, too. While you don’t control the results you receive, you do control the search query that you type into google.com. Moreover, search engines have an incentive to find results that are relevant to your search query.

With that in mind, we can add some exclusions to the definition:

SOCIALLY DISTRIBUTES CONTENT.—

(A) IN GENERAL.—The term “socially distributes content” means means making decisions about which content to distribute to a user, where such decisions use the user’s social relations with other users, such as other users that the user follows or is friends with.

(B) DIRECT DISTRIBUTION EXCLUDED.—The term “socially distributes content” does not include, notwithstanding spam filtering, distributing the content of a user directly to recipients or subscribers, such as the recipients of an email, the members of a group chat, or the subscribers of a newsletter.

(C) SEARCH EXCLUDED.—The term “socially distributes content” does not include providing content to a user when the user deliberately and independently searches for, or specifically requests, content.

Here, the language for the search exception is based on language found in KOSA, which has a similar exception for its duty of care.

4. Engagement Data

At this point, the definition looks like it may finally be sufficient. It’s certainly hard to think of a negative example that is included in the definition.

Nonetheless, it may still be beneficial to harden this definition a bit more. And for all this talk of social media addiction and kids spending hours each day on social media, the definition does not include something that is closely tied to that problem.

In addition to looking at how social networks are used in content distribution, we can also look at how a user’s engagement with content is used in content distribution:

SOCIALLY DISTRIBUTES CONTENT.—

(A) IN GENERAL.—The term “socially distributes content” means means making decisions about which content to distribute to a user, where such decisions use—

(i) the user’s social relations with other users, such as other users that the user follows or is friends with; and

(ii) the user’s engagement or interest with content from other users, such as viewing, liking, reposting, or replying to content.

Instead of targeting a specific feature or a specific type of algorithm that is used to drive engagement, we target the fuel that powers these features and algorithms: engagement data.7 This approach offers a good blend of specificity and flexibility.

All three Vs are present. Volume comes from distributing content to millions of users. Variety comes from a distribution model that is hyper-personalized in nature. Velocity comes from the algorithms that constantly process new data about the user’s activities.

With millions of users, a tech company has limited bandwidth to address problems that affect a single user. With a hyper-personalized distribution model, though, many of the problems are also hyper-personalized in nature.

E. Liability Applies at the Top of the Stack

Even for a single site, many companies often play a role in running that site. Beneath the surface of a social media site, there’s a complex technical stack of infrastructure—with different companies handling different parts.

To store billions of posts, a social media site may rely on a cloud storage provider like Amazon Web Services. To ensure that hackers cannot take down that site with a DDoS (distributed denial of service) attack, the site may rely on DDoS protection from Cloudflare. And of course, an ISP like Comcast delivers that content to users.

In many cases, liability should only be applied at the “top of the stack.” Facebook should be held accountable for its actions, but liability should not apply to ISPs or the companies that provide technical infrastructure for social media sites. In addition to defining a “social media platform,” let’s also define a “social media company”:

SOCIAL MEDIA COMPANY.—

(A) IN GENERAL.—The term “social media company” means a person or entity that provides a social media platform.

(B) TECHNICAL INFRASTRUCTURE EXCLUDED.—The term “social media company” does not include a person or entity acting in its capacity as a provider of—

(i) a common carrier service subject to the Communications Act of 1934 (47 U.S.C. 151 et seq.) and all Acts amendatory thereof and supplementary thereto;

(ii) a broadband internet access service (as such term is defined for purposes of section 8.1(b) of title 47, Code of Federal Regulations, or any successor regulation); or

(iii) an interactive computer service that is used by a social media platform for the management, control, or operation of that social media platform, including for services such as web hosting, domain registration, content delivery networks, caching, security, back-end data storage, and cloud management.

The first two exceptions were copied from KOSA’s definition of “covered platform.” The third exception was largely copied from the Internet PACT Act; it’s not every day that we find a bill that mentions content delivery networks and caching.

And again, this exception is content-neutral. This is not a content-based question about what type of content the social media platform contains; it is a content-neutral question about where regulation occurs.

III. The Full Definition

Now that we have all the pieces of our definition, let’s see the whole product, including the findings from part I; the definitions logically flow from the findings.

SEC. _. FINDINGS

The Legislature finds the following:

(1) The State has a compelling interest in protecting the physical and psychological well-being of minors.

(2) The Internet is not a monolithic medium but instead contains many distinct mediums, such as social media, search, and e-commerce.

(3) Existing measures to protect minors on social media have been insufficient for reasons including—

(A) the difficulty of content moderation at the scale of a platform with millions of user-generated content providers;

(B) the difficulty of making subjective judgments via algorithms, such as identifying content that harms the physical or psychological well-being of minors; and

(C) limited interoperability between social media platforms and third-party child safety tools, in part due to privacy concerns about sharing user data with third parties.

(4) Social media companies have failed to control the negative impacts of their algorithms to distribute content for reasons including—

(A) the scale of a platform with millions of users, combined with the personalized nature of content distribution;

(B) the natural incentive of such companies to maximize engagement and time spent on their platforms; and

(C) the limited degree of control that users have over the content they receive.

(5) Limited accountability exists on social media platforms for bad actors, especially given the anonymous or hard-to-track nature of many such actors.

(6) Users frequently encounter sexually explicit material accidentally on social media.

(7) Social media platforms are accessible—

(A) from a wide variety of devices, ranging from an individual’s smartphone to a laptop at a friend’s house to a desktop in a public library; and

(B) via a variety of methods on a single device, including apps and websites.

SEC. _. DEFINITIONS

In this Act:

(1) DAILY ACTIVE CONTENT PROVIDERS.—The term “daily active content providers” means the number of users who serve as an information content providers during a single day.

(2) INFORMATION CONTENT PROVIDER.—

(A) IN GENERAL.—The term “information content provider” has the meaning given the term in section 230(f)(3) of the Communications Act of 1934 (47 U.S.C. 230(f)(3)).

(B) SECONDARY CONTENT EXCLUDED.—The term “information content provider,” with respect to an interactive computer service, does not apply to content that depends on other content on the interactive computer service, such as comments on an article, reviews for a product, or replies to a post.

(C) COMMERCIAL CONTENT EXCLUDED.—The term “information content provider,” with respect to an interactive computer service, does not apply to content that is designed by the interactive computer service to facilitate commerce, such as product listings, available drivers for ridesharing, or booking information for accommodations.

(3) INTERACTIVE COMPUTER SERVICE.—The term “interactive computer service” has the meaning given the term in section 230(f)(2) of the Communications Act of 1934 (47 U.S.C. 230(f)(2)).

(4) SOCIAL MEDIA COMPANY.—

(A) IN GENERAL.—The term “social media company” means a person or entity that provides a social media platform.

(B) TECHNICAL INFRASTRUCTURE EXCLUDED.—The term “social media company” does not include a person or entity acting in its capacity as a provider of—

(i) a common carrier service subject to the Communications Act of 1934 (47 U.S.C. 151 et seq.) and all Acts amendatory thereof and supplementary thereto;

(ii) a broadband internet access service (as such term is defined for purposes of section 8.1(b) of title 47, Code of Federal Regulations, or any successor regulation); or

(iii) an interactive computer service that is used by a social media platform for the management, control, or operation of that social media platform, including for services such as web hosting, domain registration, content delivery networks, caching, security, back-end data storage, and cloud management.

(5) SOCIAL MEDIA PLATFORM.—The term “social media platform” means an interactive computer service that—

(A) is directed to a general audience, notwithstanding whether content is delivered via text, images, audio, video, or other types of media content;

(B) has averaged at least 1,000,000 daily active content providers over the previous 180 days; and

(C) socially distributes content.

(6) SOCIALLY DISTRIBUTES CONTENT.—

(A) IN GENERAL.—The term “socially distributes content” means means making decisions about which content to distribute to a user, where such decisions use—

(i) the user’s social relations with other users, such as other users that the user follows or is friends with; and

(ii) the user’s engagement or interest with content from other users, such as viewing, liking, reposting, or replying to content.

(B) DIRECT DISTRIBUTION EXCLUDED.—The term “socially distributes content” does not include, notwithstanding spam filtering, distributing the content of a user directly to recipients or subscribers, such as the recipients of an email, the members of a group chat, or the subscribers of a newsletter.

(C) SEARCH EXCLUDED.—The term “socially distributes content” does not include providing content to a user when the user deliberately and independently searches for, or specifically requests, content.

(7) USER.—The term “user” means, with respect to a social media platform, an individual who registers an account or creates a profile on the social media platform.

Of course, some legal details may need to be ironed out, but Congress and most state legislatures have a service known as Legislative Counsel, which handles the legal details of drafting legislation.

In fact, Legislative Counsel can work with things that are far less structured than draft legislative text. Part of their job is to take legislative proposals written in plain English and translate them to legislative text; this helps support a legislature whose representatives are teachers, doctors, and engineers—not just lawyers.

Subscribe now

0-5. If your football team went 0-5 (be it a professional or college team), it would be time to shake things up. State legislatures who have passed age verification laws for social media have gone 0-5 in legal challenges.1 Fortunately, this engineer—who has also served as a tech fellow in Congress—has already been figuring out how to fix it.

I. 0-5 vs. 2-0

Additionally, state legislatures have gone 0-5 when it comes to writing a content-neutral definition of social media; the court has ruled that every definition thus far is content-based.2 If you’re not a nerd, why does that detail matter? What are the stakes?

A. Content-Based vs. Content-Neutral

In a First Amendment challenge, courts will look at both what the law does, and who the law applies to.3 The definition of social media matters because it determines who the law applies to. As a commonsense example, the courts would be extremely skeptical of a neutral-sounding law that curiously only applied to X/Twitter, but not to other social media platforms; it’s a thinly-veiled attempt to target Elon Musk.

In the usual First Amendment case, the court will determine if the law is content-based or content-neutral. As the Supreme Court said, “As a general rule, laws that by their terms distinguish favored speech from disfavored speech on the basis of the ideas or views expressed are content based.” In practice, the courts can be very picky—in part because they’re very protective of free speech.

That leads into the second question of stakes: what’s the difference between a content-neutral and a content-based law? A content-neutral law is subject to intermediate scrutiny, while a content-based law is subject to strict scrutiny. Strict scrutiny, however, tends to be “strict in theory, fatal in fact”; 0-5 is definitely fatal in fact.

B. Nothing New Under the Sun

These lawsuits have all been filed by NetChoice, a trade association that lobbies for Big Tech. Every time NetChoice wins, they try to manufacture a narrative that history will repeat itself if anyone else tries to regulate social media. While “those that fail to learn from history are doomed to repeat it,” those who learn from history are not consigned to the same fate as their predecessors.

History did not begin with social media, either. At one point in time, cable was the new technology and the new forum for expression. Back then, cable companies started dropping broadcast channels, such as the local NBC station—despite the fact that these channels were popular with consumers. Congress stepped in and passed must-carry, which forced cable companies to carry local broadcast stations.4

In response—and tell me if you’ve heard this one before—the cable industry sued, claiming that must-carry violated their free speech rights. That battle, Turner Broadcasting System v. FCC, reached the Supreme Court twice, but the government went 2-0; must-carry survived intermediate scrutiny, as it was a content-neutral law.5

And even though social media is a very different medium for cable, the same First Amendment principles apply to both. There is nothing new under the sun.

C. The Medium is the Problem

Could we protect kids on social media if tech companies would just do a better job of content moderation? According to social psychologist Jonathan Haidt (author of the #1 New York Times bestseller The Anxious Generation), the answer is a firm no: “Social media is just not appropriate for children.” Haidt also frames the issue another way: “The medium is the problem.”

That’s not just good policy advice; it’s also good legal advice. In Turner I, the Supreme Court said that medium-based distinctions are often content-neutral: “It would be error to conclude, however, that the First Amendment mandates strict scrutiny for any speech regulation that applies to one medium (or a subset thereof) but not others.”6 And as an oft-quoted line from Southeastern Promotions v. Conrad (1975) goes, “Each medium of expression . . . must be assessed for First Amendment purposes by standards suited to it, for each may present its own problems.”

The Internet is not a monolithic medium. It contains many distinct mediums, such as social media, search, and e-commerce. Two oft-cited precedents, Reno v. ACLU (1997) and Ashcroft v. ACLU (2004), dealt with laws from the 1990s that tried to regulate the entire Internet: the Communications Decency Act of 1996 (CDA) and the Child Online Protection Act of 1998 (COPA). Today, however, nobody is proposing that we age-gate access to the entire Internet. Age verification is only being proposed for mediums that pose heightened risk to children, such as social media and pornographic sites.

The difference between cable and social media, however, is that social media is much harder to define than cable. Defining the medium is the problem.7

II. Change the Text, Change Your Fate: Findings

Legal analysis of a law begins with the text of the law—especially for a judge with a textualist philosophy. To change the fate of a law, one only needs to change its text. The challenge lies in figuring out how to change it.

A. Why Findings Matter

While knowing and citing the Turner cases is a good start, an even better approach is to do the same things that Congress did when it passed must-carry.

In particular, the “unusually detailed statutory findings” played a role in persuading the courts to apply intermediate scrutiny in Turner I. Findings have the power to persuade, but they don’t have the power to control. Courts will not believe that something is true just because the findings say it’s true, but well-written findings can have a very strong persuasive effect.

When writing these findings, you have to remember who the audience is: the courts. When a judge conducts a First Amendment analysis of a law, these findings are designed to help answer the questions they will ask. The findings need to be written with that specific audience and that specific purpose in mind.

If, per Turner I, regulations can be “justified by the special characteristics” of the medium, then someone has to explain the special characteristics of social media.

B. Intermediate Scrutiny

Likewise, you need to know how intermediate scrutiny operates if you want your legislation to survive intermediate scrutiny.

Intermediate scrutiny has two parts. First, the legislation needs to further an important or substantial government interest. (Under strict scrutiny, it has to be a compelling government interest.)

Second, the legislation needs to be narrowly tailored. Unlike strict scrutiny, the government does not have to use the least restrictive means, but to quote Ward v. Rock Against Racism (1989), the government still cannot “burden substantially more speech than is necessary to further the government's legitimate interests.” Nonetheless, Turner II also did establish that under intermediate scrutiny, the government does get to decide the degree to which it will promote its interests:

C. Tell Your Story Without Experts

A “war of experts” against Big Tech is a dicey proposition. With their deep pockets, these companies will easily have the resources to find (or pay) experts who can manufacture their preferred narrative. And if a judge with limited expertise has a hard time telling which experts are right, that small army of experts that Big Tech can summon may appear more persuasive—regardless of what actually is true.

Expertise is important, but you must first tell your story without experts. Often, good findings promote an intuitive narrative that is reasonably persuasive to non-experts.

Most importantly, that narrative sets the anchor before we consult the experts. Of course, that anchor will look unreasonable if the evidence is one-sided against you when we do consult the experts. But if a judge with limited expertise has a hard time telling which experts are right, they may default to the anchor.

This strategy also aligns with the question that judges ask for intermediate scrutiny. Per Turner II, “The question is not whether Congress, as an objective matter, was correct . . . Rather, the question is whether the legislative conclusion was reasonable and supported by substantial evidence in the record before Congress.” First, you set a reasonably persuasive anchor. Then, you provide evidence to hold that anchor.

As an added bonus, under intermediate scrutiny, judges give more deference to the legislature’s judgment when they resolve conflicting evidence: “The Constitution gives to Congress the role of weighing conflicting evidence in the legislative process.”

Of course, experts also have a role in finding the story to tell. A great narrative needs to be backed by great facts; you can’t pick a narrative based on personal whims and then ask experts to manufacture the facts to back that narrative. And in many cases, a good finding will use an expository tone and make a straightforward statement of fact.

III. Model Findings with Commentary

The Legislature finds the following:

(1) The State has a compelling interest in protecting the physical and psychological well-being of minors.

(2) The Internet is not a monolithic medium but instead contains many distinct mediums, such as social media, search, and e-commerce.

(3) Existing measures to protect minors on social media have been insufficient for reasons including—

(A) the difficulty of content moderation at the scale of a platform with millions of user-generated content providers;

(B) the difficulty of making subjective judgments via algorithms, such as identifying content that harms the physical or psychological well-being of minors; and

(C) limited interoperability between social media platforms and third-party child safety tools, in part due to privacy concerns about sharing user data with third parties.

(4) Social media companies have failed to control the negative impacts of their algorithms to distribute content for reasons including—

(A) the scale of a platform with millions of users, combined with the personalized nature of content distribution;

(B) the natural incentive of such companies to maximize engagement and time spent on their platforms; and

(C) the limited degree of control that users have over the content they receive.

(5) Limited accountability exists on social media platforms for bad actors, especially given the anonymous or hard-to-track nature of many such actors.

(6) Users frequently encounter sexually explicit material accidentally on social media.

(7) Social media platforms are accessible—

(A) from a wide variety of devices, ranging from an individual’s smartphone to a laptop at a friend’s house to a desktop in a public library; and

(B) via a variety of methods on a single device, including apps and websites.

Finding 1

(1) The State has a compelling interest in protecting the physical and psychological well-being of minors.

Don’t reinvent the wheel.

Sable Communications v. FCC (1989): “We have recognized that there is a compelling interest in protecting the physical and psychological wellbeing of minors.”

Additionally, “psychological well-being” is a much better framing than “harmful content.” Under the framing of harmful content, the obvious counterargument is that while some harmful content exists on social media, most content is not harmful; the legislation is extremely overbroad because it targets social media as a whole.

The framing of psychological well-being plays out much differently. Consider the story of 16-year-old Chase Nasca, who committed suicide after TikTok showed him over 1,000 unsolicited videos of violence and suicide. Does it really matter whether those 1,000 videos were 5% or 50% of the content that Chase saw? What really matters is that those videos—regardless of the percentage—led Chase to commit suicide.

Finding 2

(2) The Internet is not a monolithic medium but instead contains many distinct mediums, such as social media, search, and e-commerce.

Medium-based distinctions are content-neutral.

This finding is fairly intuitive and straightforward; you don’t have to be an expert to know that the Internet is not a monolithic entity. It also distinguishes a social media law from the CDA in Reno and COPA in Ashcroft II. Both the CDA and COPA tried to regulate the entire Internet, not a specific medium like social media.

Finding 3

(3) Existing measures to protect minors on social media have been insufficient for reasons including—

(A) the difficulty of content moderation at the scale of a platform with millions of user-generated content providers;

(B) the difficulty of making subjective judgments via algorithms, such as identifying content that harms the physical or psychological well-being of minors; and

(C) limited interoperability between social media platforms and third-party child safety tools, in part due to privacy concerns about sharing user data with third parties.

Content moderation at scale is hard.

You may be able to detect an engineer’s influence in crafting this finding, but you don’t need to be an engineer to know that content moderation is hard when millions of users are producing content every day.

At that scale, you inevitably will have to rely on algorithms more and more; humans alone can’t handle that volume of content. But how effective are these algorithms when they have to make very subjective judgments, such as whether content would harm the psychological well-being of a child?

Even the advent of AI is not a magical panacea. To be clear, there are many objective tasks that AI handles well, such as image recognition for handwritten digits or for traffic signs. But if our self-driving cars hallucinated as often as ChatGPT did, they would swiftly be taken off the roads.8

To further complicate matters, social media sites often operate as closed ecosystems. There’s a two-word explanation for why Facebook is understandably wary about sharing user data with third parties: Cambridge Analytica. (It’s also worth nothing that Cambridge Analytica obtained personal data via an external researcher who claimed to be collecting it for academic purposes.)

At the end of the day, you can certainly understand why Haidt arrived at the conclusion he did: “Even if social media companies could reduce sextortion, CSAM, deepfake porn, bullying, self-harm content, drug deals, and social-media induced suicide by 80%, I think the main take away from those Senate hearings is: Social media is just not appropriate for children.” The medium is the problem.

Finding 4

(4) Social media companies have failed to control the negative impacts of their algorithms to distribute content for reasons including—

(A) the scale of a platform with millions of users, combined with the personalized nature of content distribution;

(B) the natural incentive of such companies to maximize engagement and time spent on their platforms; and

(C) the limited degree of control that users have over the content they receive.

The distribution model matters.

In defining social media, we will have to consider many “negative examples” of sites that aren’t social media: the comments section of the New York Times, Netflix, Wikipedia, Substack, etc. A social media law should not apply to these sites—especially since overbreadth can be fatal in a First Amendment challenge.

One underexamined but vitally important (and content-neutral) difference is the distribution model. Simply put, you’re not going to see over 1,000 unsolicited videos of violence and suicide if you subscribe to some newsletters on Substack.

When you have millions of users, you have limited bandwidth to address problems affecting only a single user. The highly personalized nature of content distribution on social media, however, means that many problems are also personalized in nature.

In particular, social media platforms—who have natural incentives to maximize engagement (especially when more engagement leads to more ad revenue)—have a wealth of personal engagement data. Their algorithms can use the content that you have viewed, liked, reposted, replied to, etc., to decide what content to serve you.

Subparagraph (C) of this finding also alludes to another important aspect of the distribution model: lack of control. If you don’t like a Substack newsletter, you can easily unsubscribe from it. If TikTok’s algorithms start feeding you suicide content or eating disorder content, however, your options to make it go away are more limited.

This subparagaph is also written as a callback to Section 230 (the law that says that online platforms are generally not liable for the third-party content they host). Section 230 included this finding: “(2) These [interactive computer services] offer users a great degree of control over the information that they receive, as well as the potential for even greater control in the future as technology develops.” Social media has not lived up to that potential. The world has changed since Section 230 was enacted in 1996.

Finding 5

(5) Limited accountability exists on social media platforms for bad actors, especially given the anonymous or hard-to-track nature of many such actors.

Use a frame of reference that courts are familiar with.

While social media can be somewhat new and unfamiliar to the courts, they do have more extensive experience with older mediums such as broadcast and cable.

In the world of broadcast, if CBS broadcasted sexually explicit material during the day (or the breast of Janet Jackson during the Super Bowl), they could expect a fine from the FCC. And while the FCC regulations do not apply to cable, the incentives of that medium make it highly unlikely, for example, that ESPN’s Pardon the Interruption would interrupt your sports viewing experience with hardcore pornography.

The same set of incentives simply do not exist for content producers on social media. At worst, your account could be banned, but you can often create a new account. Even if Instagram investigates a “sextortion” case on their platform, what can you do when—in the case of Walker Montgomery—they trace the account’s IP address to Nigeria?

When the Supreme Court compared broadcast regulations to dial-a-porn regulations in Sable Communications v. FCC (1989), they noted that while an “unexpected outburst on a radio broadcast” tends to be “invasive or surprising,” dial-a-porn is different: “In contrast to public displays, unsolicited mailings, and other means of expression which the recipient has no meaningful opportunity to avoid, the dial-it medium requires the listener to take affirmative steps to receive the communication.”

As for social media, a sextortion attempt is often unsolicited and invasive in nature. Problems on social media are often caused by unsolicited or invasive content— especially when users have a limited degree of control over the content they receive.

Finding 6

(6) Users frequently encounter sexually explicit material accidentally on social media.

This is self-evident to anyone with an X/Twitter account.

This finding is a direct callback to Reno: “Though [sexually explicit] material is widely available, users seldom encounter such content accidentally.” That may have been true for the Internet of 1997, but it’s definitely not true for the Internet of 2024.

Finding 7

(7) Social media platforms are accessible—

(A) from a wide variety of devices, ranging from an individual’s smartphone to a laptop at a friend’s house to a desktop in a public library; and

(B) via a variety of methods on a single device, including apps and websites.

Do you try to cut kids off at every possible path, or cut them off at the destination?

This is another example of a finding that makes straightforward statements of facts in an expository tone, but which also sets up the narrative.

In the early days of the Internet, many households would have had a single desktop in a common area of the house, and any online content would be accessed via a web browser. Today, most kids have a smartphone that travels everywhere with them.

Some claim parental controls are the answer, but even if you set up perfect parental controls on a single device—a task easier said than done—what if the kid uses a different device? An old smartphone (or a cheap smartphone the kid bought) would not have talk, text, or data, but it would have Internet access wherever there’s WiFi.9 The kid could also use a laptop at a friend’s house or a desktop at a public library.

And even on a single device with parental controls, the task is not straightforward. Perhaps you blocked the Facebook app, but did you block Facebook’s website? And what if the kid downloads a proxy app and uses that to browse Facebook?

Age verification, by contrast, is applied at the destination. It doesn’t matter which device the kid uses to access Instagram, or whether they accessed Instagram via a browser or via an app; they still need to verify their age to create an account.

Cutting kids off at the destination offers a greater degree of protection, compared to trying to cut them off at each possible path they could take.10 Parental controls are not a narrowly tailored alternative to age verification. To quote Turner II, “In the final analysis this alternative represents nothing more than appellants’ ‘ “[dis]agreement with the responsible decisionmaker concerning” . . . the degree to which [the Government’s] interests should be promoted.’ ”

Now that we have model findings, the next step is to write a model definition of social media. Ideally, the definition should naturally flow from the findings, and it should codify the special characteristics of social media that we identified in the findings. In the next part, we’ll dive into the brass tacks of writing that definition.

Subscribe now

Social media may be a forum for expression, but that doesn’t mean that any bill that touches social media is unconstitutional. Cable is also a forum for expression, but despite that, the Supreme Court upheld the FCC’s must-carry rules—rules that force cable companies to carry local broadcast stations.

The dispute over must-carry, Turner Broadcasting System v. FCC, was settled in two separate cases. In Turner I (1994), the Supreme ruled that must-carry is content-neutral and only subject to intermediate scrutiny. In Turner II (1997), the Supreme Court ruled that must-carry survives intermediate scrutiny.

As for the TikTok bill, there is nothing new under the sun. In part I, we showed that the TikTok bill is content-neutral and only subject to intermediate scrutiny. In part II, we’ll show that this bill survives intermediate scrutiny. With the TikTok bill on the verge of becoming a law, some will use legal disputes to relitigate the policy disputes that they lost in Congress. That didn’t work in Turner II, and it won’t work here. The courts won’t replace Congress’s policy judgments with its own policy judgments.

Intermediate Scrutiny: How It Works

What is intermediate scrutiny? Here is the definition from Turner II:

Turner II did not just establish the criteria for intermediate scrutiny, though; it also establishes how courts evaluate those criteria. While some treat the First Amendment like a magical trump card that overturns laws, that’s not how intermediate scrutiny works (though it could be an accurate description of how strict scrutiny works).

Before we explain the details of how intermediate scrutiny works, let’s first explain why it creates a more favorable legal environment for the government.

First, content-neutral laws pose less of a threat to free speech: “Content-neutral regulations do not pose the same ‘inherent dangers to free expression’ that content-based regulations do, and thus are subject to a less rigorous analysis, which affords the Government latitude in designing a regulatory solution.” Second, the courts are not a policy-making institution; Congress is the policy-making institution. This is separation of powers 101: “We are not at liberty to substitute our judgment for the reasonable conclusion of a legislative body.”

(The courts, however, will assert their judicial power on constitutional questions—including the question of whether a law is content-based or content-neutral. Nonetheless, if a court—after carefully scrutinizing a law—does determine that a law is content-neutral, the legal landscape will become more favorable to the government.)

Since the courts will defer to Congress on policy matters, under intermediate scrutiny, the courts will not make their own policy judgment as to which side had the better evidence. They will instead ask if Congress had substantial evidence: “Our sole obligation is ‘to assure that, in formulating its judgments, Congress has drawn reasonable inferences based on substantial evidence.’ ”

This core principle also applies when conflicting evidence exists: “The Constitution gives to Congress the role of weighing conflicting evidence in the legislative process.” So long as Congress’s conclusion was reasonable and supported by substantial evidence, the existence of other possible conclusions does not negate Congress’s judgment—even if those other conclusions were also reasonable.

Moreover, the courts have recognized that Congress’s authority to make policy judgments includes the authority to make predictive judgments: “A fundamental principle of legislation is that Congress is under no obligation to wait until the entire harm occurs but may act to prevent it.”

The First Amendment operates differently under intermediate scrutiny. The concluding sentences of Turner II concisely state what it does (and does not) require:

Important Governmental Interests

Does TikTok pose a national security risk? When he appeared on 60 Minutes, former CIA officer Klon Kitchen framed the issue perfectly:

Imagine you woke up tomorrow morning and you saw a news report that China had distributed 100 million sensors around the United States, and that any time an American walked past one of these sensor, this sensor automatically collected off of your phone your name, your home address, your personal network, who you're friends with, your online viewing habits and a whole host of other pieces of information. Well, that's precisely what TikTok is. It has 100 million U.S. users, it collects all of that information.

When Congress is allowed to make predictive judgments, and it only needs to show that it acted reasonably based on substantial evidence, it should have an easy time proving the first prong: that the bill advances an important government interest.

When it comes to the threat posed by TikTok, there certainly won’t be a shortage of evidence, either. On the right, FCC Commissioner Brendan Carr has compiled an excellent Twitter thread of evidence. On the left, Sen. Maria Cantwell (D-WA) and Sen. Mark Warner (D-VA) engaged in a colloquy on the Senate floor that also laid out the substantial evidence for the TikTok bill.

Critics, on the other hand, frequently ignore this evidence when they try to argue the contrary. For example, even after TikTok was caught redhanded spying on journalists, Jennifer Huddleston and Paul Matzke of the Cato Institute still claim that the evidence for the TikTok bill is only “mere suspicion that TikTok might someday be used to monitor American citizens.” This is not a serious argument.

Writing for Lawfare, Adam Chan also notes that the US government has a very strong case on this first prong. In particular, he cites precedents relating to national security (Holder v. Humanitarian Law Project (2010)) and foreign influence in elections (Bluman v. FEC (2011)) where the Supreme Court even upheld content-based laws.

Narrow Tailoring

Realistically, the question of whether the TikTok bill can survive intermediate scrutiny is going to center on the second prong: narrow tailoring. As Turner II noted, narrow tailoring under intermediate scrutiny is very different from strict scrutiny: “we will not invalidate the preferred remedial scheme because some alternative solution is marginally less intrusive on a speaker’s First Amendment interests.”

In Turner II, the Supreme Court also evaluated the effectiveness of proposed alternatives to must-carry. For example, it rejected a leased-access regime as a narrowly tailored alternative to must-carry, as “it would not be as effective in achieving Congress’ further goal of ensuring that significant programming remains available for the 40 percent of American households without cable.”

As a corollary, under intermediate scrutiny, the government decides to what degree it will promote its legitimate interests: “the validity of its determination ‘ “does not turn on a judge’s agreement with the responsible decisionmaker concerning” . . . the degree to which [the Government’s] interests should be promoted.’ ”

So long as Congress “does not burden substantially more speech than necessary,” the courts will defer to the policy judgments of Congress.

Divest First, Ban Second

With the TikTok bill, divestment is the first option; a ban is the second option. This strategy has significant legal consequences. If a ban was the first option, perhaps one could argue that it burdens more speech than necessary—and that divestment is a narrowly tailored alternative. You can’t make that same argument if the Chinese Communist Party refuses to divest, though. At that point, a ban is necessary.

The Real Problem: The Chinese Communist Party

Are there narrowly tailored alternatives to divestment? Since the real problem with TikTok is the Chinese Communist Party, the answer is no.

Some have tried to narrowly portray the problem as a privacy issue, but the problems created by the CCP’s control of TikTok exist along multiple dimensions: privacy, child safety, espionage, and foreign interference in elections, among others.

With some dimensions, such as privacy, the problem is the CCP’s access to US user data. With other dimensions, such as child safety, the problem is the CCP’s control of the algorithms. For example, 16-year-old Chase Nasca committed suicide after TikTok’s algorithms fed him over 1,000 unsolicited videos promoting violence and suicide. While every social media platform has child safety issues, TikTok is the worst of the worst. The Chinese Communist Party does not care about dead American kids.

Thus, a national privacy law is not a narrowly tailored alternative, as it only deals with one dimension: privacy. It won’t fix the algorithms that are killing our kids.

And even if the only problem with TikTok was privacy, a national privacy law would still not be a narrowly tailored alternative for another reason. While a national privacy law would be an effective deterrent for a truly private company like Facebook, it would not be an effective deterrent for the CCP. After all, we already have intellectual property (IP) laws on the books, but that has not deterred IP theft from China.

Project Texas

Finally, some have suggested that a narrowly tailored alternative is Project Texas, which would put US user data in a lockbox located in the USA. Speaking as an engineer, this option is not viable.

The short answer here is that you can only trust Project Texas if you trust the CCP; it suffices to say that you cannot trust the CCP. Nonetheless, here is the long answer.

Under Chinese law, if the lockbox is located in China, or if it is owned by a Chinese company, the CCP can make you open the lockbox. TikTok’s parent company, ByteDance, is a Chinese company that must obey Chinese law. Putting the lockbox in America does not fully solve the problem. If ByteDance still has a key to the lockbox, it does not matter whether the lockbox is located in China or America.

Moreover, as an engineer, if you asked me to do a threat model for a lockbox, I would ask you, “Who designed the lockbox?” If the answer is a Chinese company, that’s your critical vulnerability right there. Since ByteDance controls TikTok, Project Texas is a lockbox that is designed by China. Lockbox rejected.

One consultant working on Project Texas said, “I feel like with these tools, there’s some backdoor to access user data in almost all of them, which is exhausting.” Reporting from the Wall Street Journal likewise confirmed that Project Texas was a “porous” system that does not live up to its promises: “Employees say ByteDance managers continue to request U.S. data.”

(It's not just ByteDance, either. Backdoors have been found on devices from Lenovo, a Chinese company, as far back as 2013. Telecom equipment by Huawei, a Chinese company, had backdoors that let them covertly access US telecom networks.)

Even if ByteDance did not have a key to the lockbox, what if an American employee removes data from the lockbox and shares it with ByteDance? One TikTok employee had a manager in Seattle on paper, but actually reported to a ByteDance executive in Beijing: “Nearly every 14 days . . . he emailed spreadsheets filled with data for hundreds of thousands of U.S. users to ByteDance workers in Beijing.”

In cybersecurity, you often assume vulnerabilities will be exploited; even if you can’t find the exploit, someone else will. Project Texas's vulnerabilities are downstream of the real vulnerability: it’s designed by China. It’s foolish to assume that China will not exploit that. Once again, the real problem is the Chinese Communist Party.

If the Chinese Communist Party refuses to divest TikTok, TikTok delenda est.

For example, the Supreme Court upheld the FCC’s must-carry rules—rules that force cable companies to carry local broadcast stations—even though cable is another forum for expression protected by the First Amendment. Likewise, the TikTok bill, which would force the Chinese Communist Party (CCP) to divest TikTok—and would ban TikTok if the CCP refuses to divest—does not violate the First Amendment.

In part I, we’ll cover two key distinctions: between using TikTok and owning TikTok, and between a content-based and content-neutral law. The TikTok bill would still let the Chinese Communist Party use TikTok, but it would restrict their ability to own and control TikTok. Moreover, since the TikTok bill distinguishes between social media platforms based on foreign ownership—not based on the speech on the platform—it is a content-neutral bill only subject to intermediate scrutiny.

We’ll also cover how this bill differs from two past attempts to regulate TikTok: then-President Trump’s executive order to ban TikTok in 2020, and a Montana law that banned TikTok in 2023. Trump’s efforts failed because he did not have the legal authority to ban TikTok—not because he violated the First Amendment. Montana’s efforts failed because foreign policy is the federal government's exclusive domain. The obvious solution to both problems is a federal bill that would force TikTok to divest—and then ban TikTok if they refuse to divest.

Using TikTok vs. Owning TikTok

While the CCP has the right to use TikTok under the First Amendment, it does not have the “right” to own TikTok under the First Amendment.

During the Cold War, the Supreme Court ruled in Lamont v. Postmaster General (1965) that the Soviet Union had a First Amendment right to distribute “communist political propaganda” via the United States Postal Service (USPS). Likewise, the CCP today likely has a right to create a TikTok account and use it to spread its propaganda.

Critics such as Jameel Jaffer of the Knight Institute, however, have cited Lamont to make a very different argument: that the CCP has the right to own and control TikTok. Lamont said that a Soviet company could use the USPS; it never said that a Soviet company could own the USPS.

There’s no private sector in communist China. TikTok’s parent company, ByteDance, is a Chinese company. Under Chinese law—such as the 2017 National Intelligence Law—ByteDance must obey the dictates of the CCP. The lines that divide the public and private sectors in America do not exist in China. People who even criticize the CCP, such as Chinese tech icon Jack Ma, have a habit of mysteriously “disappearing.”

And in America, one TikTok employee, Evan Turner, was “assigned” to a manager in Seattle—a manager he never met—but actually reported to a ByteDance executive in Beijing whom he met with weekly. As Fortune reported, “Nearly every 14 days, as part of Turner’s job throughout 2022, he emailed spreadsheets filled with data for hundreds of thousands of U.S. users to ByteDance workers in Beijing.”

As Alec Stapp of the Institute for Progress (and many others) have pointed out, it would be unthinkable to let the Soviet Union own ABC, NBC, or CBS during the Cold War. So why would we let the Chinese Communist Party control TikTok today?

And to that point, the US does have laws restricting foreign ownership of mass communications media such as radio and broadcast—laws that have not been struck down as unconstitutional. A similar law restricting foreign ownership of social media—another mass communications media—would also be constitutional.

Content-Based vs. Content-Neutral

When critics claim a bill violates the First Amendment, they often use the following template. First, they argue that since the bill regulates speech, it is subject to strict scrutiny. Then, they argue that the bill does not survive strict scrutiny—an easy argument since strict scrutiny is often “strict in theory, fatal in fact.”

When the Supreme Court upheld the FCC’s must-carry rules, however, they applied intermediate scrutiny to those rules. While content-based laws are subject to strict scrutiny, a content-neutral law is only subject to intermediate scrutiny.

What’s the difference between a content-based law and a content-neutral law? That line is not always easy to draw, but “[a]s a general rule, laws that by their terms distinguish favored speech from disfavored speech on the basis of the ideas or views expressed are content based,” to quote Turner Broadcasting System v FCC (1994).

(There are actually two Turner cases. In Turner I (1994), the Supreme Court ruled that the FCC’s must-carry rules are subject to intermediate scrutiny. In Turner II (1997), the Supreme Court ruled that these must-carry rules also survive intermediate scrutiny.)

As the Supreme Court said in Turner II, “A content-neutral regulation will be sustained under the First Amendment if it advances important governmental interests unrelated to the suppression of free speech and does not burden substantially more speech than necessary to further those interests.” Under strict scrutiny, by contrast, the law must advance a compelling government interest, and it must use the least restrictive means to advance that interest.

The TikTok Bill is Content-Neutral

The TikTok bill is content-neutral for a simple reason: it distinguishes between social media platforms based on foreign ownership, not based on the speech on the platform.

Many critics, however, will implicitly or explicitly assume that strict scrutiny applies, even though that assumption is not warranted. The most egregious example comes from Jennifer Huddleston of the Cato Institute:

Under First Amendment precedents, the government will need to prove that forced divestment or otherwise banning of the app is both based on a compelling government interest and represents the least restrictive means of advancing that interest. In December, a federal district court enjoined a TikTok ban in Montana on First Amendment grounds as it was “unlikely to pass even intermediate scrutiny.”

Here, Huddleston claims strict scrutiny applies without providing evidence that the bill is content-based. The court case that Huddleston cites disagrees. In that case, TikTok argued Montana’s law was content-based, while Montana argued it was content-neutral. While Judge Donald Molloy did not issue a definitive decision, he did say that Montana “is closer to the legal mark.”1

Moreover, Huddleston omits a key reason why Montana’s law did not survive intermediate scrutiny: “Montana does not have constitutional authority in the field of foreign affairs.” Foreign policy is the exclusive domain of the federal government. As a result, “the law’s foreign policy purpose is not an important Montana state interest.”

In the context of Huddleston’s argument on whether the federal government has “a compelling national security interest at stake,” it makes no sense to cite a case that says nothing about that question—other than to say that national security is only a federal interest and not a state interest. Writing for Lawfare, Adam Chan also arrived at a similar conclusion: “When it comes to First Amendment analysis, virtually none of Molloy’s intermediate scrutiny analysis would likely hinder a federal ban.”

If any conclusion can be drawn from the Montana case, it is that this issue can only be solved on the federal level—where the federal government is in a good position to win.

Targeting Social Media is Content-Neutral

The TikTok bill applies not just to TikTok, but to any social media app controlled by China, Russia, Iran, or North Korea. But what about other apps that are foreign-controlled? If the bill only targets social media apps but not other apps, is that content-based? In short, no.

First, cable operators once made a similar argument; they argued that must-carry rules were content-based because they targeted cable but not other mediums of communication. The Supreme Court rejected that argument in Turner I: “It would be error to conclude, however, that the First Amendment mandates strict scrutiny for any speech regulation that applies to one medium (or a subset thereof) but not others.”2

Quoting Southeastern Promotions v. Conrad (1975), the court also said, “Each medium of expression . . . must be assessed for First Amendment purposes by standards suited to it, for each may present its own problems.” Few would doubt today that social media is a unique medium with its own problems.

Second, there is a logical reason why the bill only applies to social media. When then-President Trump tried to ban TikTok in 2020, the courts blocked that ban—not on the basis that Trump violated the First Amendment, but on the basis that he exceeded his legal authority under the International Emergency Economic Powers Act (IEEPA).3

While the IEEPA does give the President the power to deal with foreign-controlled apps in general terms, it also has a “personal communications” exception and an “information materials” exception. (The “information materials” exception is also known as the Berman Amendment.) The court ruled that both exceptions applied to TikTok—and to social media more broadly.

Since the IEEPA does not apply to social media apps, it would logically make sense to create a separate law for social media apps. The IEEPA can still be used, however, for other apps that are not subject to the “personal communications” or “information materials” exceptions; there’s no need to create a new law for those apps.

Intermediate Scrutiny (Part II)

Now that we have established that the TikTok bill is a content-neutral bill only subject to intermediate scrutiny, we must ask this question next: can the bill survive intermediate scrutiny? In short, it can, but we’ll cover that in part II…

You can read part II here.

In tech policy, you will at times inevitably wade into the realm of cryptography—an area where you often need technical expertise. Tech policy experts, however, often have a humanities degree and a policy job where they specialize in tech policy. They don’t have the right type of expertise.

Nonetheless, if you have a job that touches tech policy (such as a congressional staffer), you often have to make policy judgments—whether you have the needed expertise or not. Thus, I’ve written this crash course to help you make better judgments—and feel less lost along the way.

The goal of this crash course is not to tell you what to think; it’s to teach you how to think. (You’ll also learn why passwords should include at least eight characters, an uppercase letter, a lowercase letter, a number, a special character, etc.)

Tools, Tasks, and Protocols

Hashing, blockchain, and zero-knowledge proofs, oh my! You may have seen these terms thrown around, but what do they mean? Rather than answer that question, let’s take a step back and start with something simpler: tools, tasks, and protocols.

Let’s start with tasks. What useful thing are you trying to accomplish with technology? Here are some examples of tasks:

• Two parties send encrypted messages to each other.

• One party verifies that a document was digitally signed by another party.

• One party verifies their age for another party.

Once we establish what the task is, we design a protocol: a precise set of instructions where two or more parties communicate to accomplish a task. (A protocol is essentially a communications algorithm.)

To build a protocol, we will have various tools at our disposal. Hashing is a tool. Blockchain is a tool. Zero-knowledge proofs are a category of tools.

Tools are used to build the product; tools are not the product. In cryptography, the product we are building is a protocol.

An Example Protocol

To make these concepts more concrete, let’s design a protocol that is accessible to beginners. For this protocol, the task will be sending encrypted messages.

The Tool: Caesar Cipher

A Caesar cipher takes every letter of a message and shifts it forward in the alphabet. If you go past the end of the alphabet, you start over at A.

For example, with a shift value of 3, here is how we shift the letters:

• A becomes D: A→B→C→D

• B becomes E: B→C→D→E

• Z becomes C: Z→A→B→C

If our message is HELLO WORLD, and we use a Caesar cipher with a shift of 4, the new “message” will be LIPPS ASVPH. (For example, H becomes L: H→I→J→K→L.)

The Protocol

Protocols usually rely on a key, which is a secret value. For a Caesar cipher, the key is the shift value. Here is the protocol to encrypt messages:

Beforehand—when nobody can eavesdrop on them—Alice and Bob agree on the key: the shift value for a Caesar cipher. Let’s say that they agree on a shift of 4.

Alice sends Bob a message (or vice-versa):

1. Alice creates a message: HELLO WORLD

2. Alice shifts every letter forward 4 spots. HELLO WORLD becomes LIPPS ASVPH

3. Alice sends the encrypted message, LIPPS ASVPH, to Bob.

4. Bob shifts every letter backward 4 spots. LIPPS ASVPH becomes HELLO WORLD

If a third person, Eve, eavesdrops on the conversation, the “message” that she will see is jumbled letters: LIPPS ASVPH.

Evaluate Protocols, Not Tools

While new technology often fascinates people, sometimes we get so enamored with technology that we lose sight of the task we’re trying to accomplish. Does this new technology help us accomplish that task, or is it just a shiny new thing?

While people love playing with chatbots like ChatGPT, what happens if an airline’s chatbot gives a wrong answer about a refund policy? Air Canada learned the answer the hard way: a Canadian tribunal made them honor the refund policy. Or, as Cloudflare CEO Matthew Prince said, “AI demos are easy. AI products are hard.”

In a similar vein, if you have a colorblind friend who is holding a red ball and a green ball, you can use a zero-knowledge proof to convince him that the balls have different colors—without revealing which ball is which color. That’s a cool party trick, but it’s not clear whether this zero-knowledge proof would have practical applications.

Tools are used to build the product; tools are not the product. The thing you want to evaluate is the product, not the tools. In this case, the product is the protocol.

For cryptography in particular, you can definitely shoot yourself in the foot by misusing your tools. The protocol lets us see how we are using our tools—and whether we are misusing those tools.

Later, we will examine two very similar protocols for two very similar tasks. One protocol is secure, and one protocol is fatally flawed. Both protocols use the same tool—hashing—but one protocol misuses that tool.

The more you think at the level of protocols (and not at the level of tools), the more persuasive your arguments will be.

This insight applies in both directions, too. If you want to argue that a certain task is not possible, then what is the key challenge in designing a protocol for this task? Why are existing tools not capable of solving that challenge? (A humanities degree often will not give you the expertise needed to answer those questions.)

Evaluating the Example Protocol

As a practical example, let’s evaluate our protocol for sending encrypted messages using a Caesar cipher. It has two issues:

1. Beforehand—when nobody could eavesdrop on them—Alice and Bob agreed on the key: the shift value for the Caesar cipher. What happens when Alice and Bob cannot agree on a key beforehand?

2. There are only 25 possible keys. If Eve intercepts the encrypted message, LIPPS ASVPH, she can try to decrypt it with every possible shift value (1, 2, 3, …). Eventually, one of those shift values will work.

In the real world, we would use a different protocol for this task. And if two parties cannot agree on a key beforehand, we can use additional tools for key agreement: a way for Alice and Bob to agree on a key without revealing that key to an eavesdropper.

Negotiating the Requirements

Let’s use age verification as a case study on defining and negotiating the requirements. Here, critics will frequently raise this point: kids will find a way to bypass age verification.

That point is technically correct but practically useless. What percentage of kids bypass age verification? Is it 0.5% of kids, or 50% of kids?

When you define the requirements for a task, many requirements will not be all-or-nothing. Instead of demanding perfection, you will determine what is good enough. There’s a saying that you don’t let the perfect be the enemy of the good.

Engineers in particular typically do not talk about 100%. Instead, they talk about the “number of 9s.” For example, 99.9% would be three 9s. Amazon S3, a cloud storage service, even promises eleven 9s of durability.

So how often should an age verification system stop kids? Do we need eleven 9s? Probably not. Is one 9 (90%) a reasonable request? Yes. (Even if age verification stopped kids only 75% of the time, that would still be a major policy victory.)

That leads to a key point: there often is room for reasonable negotiation on the requirements. In some cases — especially when cryptography gets involved — a seemingly intractable technical challenge can become easily solvable if you make a reasonable concession.

Tradeoffs are common in engineering. If you asked for age verification with eleven 9s of effectiveness, it would be extremely challenging to build that in a privacy-conscious way. If you only asked for one 9, the privacy challenges become much easier to solve.

A Good Protocol: Password Authentication

As a real-world example, let’s look at one task — password authentication — and the protocol we use to accomplish this task.

The Tool: Hashing

This protocol will use one key tool: hashing. Hashing can create a “digital fingerprint” for any piece of data—such as a password, a Word document, or a video file.

• The input of a hash function is arbitrary data of any size.

• The output is a short piece of data, which is our digital fingerprint (also known as a hash or a hash value).

Here is an example where the input is a password:1

• Input: MyPassword

• Output/hash value: dc1e7c03e162397b355b6f1c895dfdf3790d98c10b920c55e91272b8eecada2a

If the input was a different password, or if the input was a large Word document, the output would be different, but it would have the same length: 64 characters.

Just like each person has a unique fingerprint, each input produces a unique hash value. Two different passwords (or two different Word documents) will never have the same hash value.2

Hashing is also a one-way operation. If all you know is the hash, it is impossible to figure out which input produced that hash — unless you get lucky and guess the input. For example, if I know the hash of your password (dc1e7c03e162397b355b6f1c895dfdf3790d98c10b920c55e91272b8eecada2a), I cannot reverse-engineer it to obtain your password (MyPassword).

The Protocol

So how does password authentication work? Here’s the basic protocol:3

A user sets/resets their password:

1. The user sends their password to a site (e.g., MyPassword).

2. The site computes the hash of that password (e.g., dc1e7c03e162397b355b6f1c895dfdf3790d98c10b920c55e91272b8eecada2a).

3. The site stores that hash in a database.

A user logs in:

1. The user sends their username and password to a site.

2. The site computes the hash of the password it just received.

3. The site retrieves the hash on file for that user.

4. If the two hashes match, the password is accepted.

If a data breach occurs, a hacker can steal the hash of your password, since that’s stored in the site’s database. But since hashing is a one-way operation, the hacker cannot reverse-engineer that hash to obtain your password.

(This protocol is missing one small yet important detail; we’ll return to that later.)

A Meta-Point on Data Breaches

Using this protocol as an example, we can also make a meta-point on data breaches: in some cases, a well-designed protocol can make certain guarantees even if a data breach occurs.

In this protocol for password authentication, your password cannot be stolen even if a data breach occurs. In my protocol for age verification, users cannot be de-anonymized even if a data breach occurs.

A Bad Protocol

Could we apply the same idea to Social Security numbers (SSNs)? Could a site use a similar protocol to store the hash of an SSN?

No. Even though we only changed one detail, the protocol is now fatally flawed. While we are using the same tool—hashing—we are now misusing that tool.

Earlier, we said, “If all you know is the hash, it is impossible to figure out which input produced that hash — unless you get lucky and guess the input.” That last part raises an intriguing possibility: instead of trying to make a lucky guess, what if you guessed every possible input? That is a brute-force attack.

Let’s say that a data breach occurred, and a hacker learns the hash of your SSN: 72de837c74b40716d430c711eebde10ff965fcc4a70c98e63a233ff36eebd6a1.4

An SSN is a 9-digit number, so there are 1 billion possible SSNs. The hacker could compute the hash of all 1 billion SSNs — until they find the SSN that matches the stolen hash. How long would that take? On my laptop, I can calculate those billion hashes in a little over a minute; the matching SSN is 123-45-6789.5

With hashing, you need to pay attention to how many possible inputs there are. If the input is 256 random bits, there are over 10⁷⁷ possible inputs: 1 followed by 77 0s.6 By comparison, there are about 10⁸⁰ atoms in the universe; a brute-force attack is impossible. With only 1 billion (10⁹) possible inputs, though, brute force will work.

The Missing Detail for Password Authentication

Our earlier protocol for password authentication was missing one key detail: the requirements for a valid password. Usually, passwords should include at least eight characters, an uppercase letter, a lowercase letter, a number, a special character, etc.

Those requirements exist because they expand the number of possible passwords, which guards against a brute-force attack. Brute force has a high chance of success for passwords under eight characters. (In practice, password length is the most important requirement. A 15-character password with only lowercase letters is much more secure than an 8-character password with all the special gadgets.)

In Summary

First, we establish what the task is (e.g., sending encrypted messages). This task will usually come with some requirements, though there often is room for reasonable negotiation on the requirements.

Next, we design a protocol. A protocol is a precise set of instructions where two or more parties communicate to accomplish a task. To build a protocol, we will have various tools at our disposal (e.g., hashing).

Tools are used to build the product; tools are not the product. The thing you want to evaluate is the protocol, not the tools. For cryptography in particular, you can definitely shoot yourself in the foot by misusing your tools. The protocol lets us see how we are using our tools—and whether we are misusing those tools.

The more you think at the level of protocols (and not at the level of tools), the more persuasive your arguments will be.