Lehigh professor calls Pennsylvania’s Character.AI lawsuit a ‘practical’ first step as Lehigh Valley lawmakers back Shapiro action

BETHLEHEM, Pa. — Pennsylvania has sued the artificial intelligence company Character.AI, alleging its chatbots posed as licensed psychiatrists and other medical professionals in violation of state law — an action Gov. Josh Shapiro’s administration says is the first of its kind announced by a U.S. governor.

The Pennsylvania Department of State filed suit against Character Technologies Inc. in Commonwealth Court on Friday and announced the action publicly Tuesday morning. The complaint alleges the company violated the state’s Medical Practice Act, and the state is seeking a preliminary injunction to halt the conduct.

The case has drawn quick reaction in the Lehigh Valley, where lawmakers in both parties have engaged with AI policy issues and where the chair of Lehigh University’s computer science department called the lawsuit a “practical” first step.

According to the complaint, an investigator from the Department of State’s Bureau of Enforcement and Investigation created a Character.AI account and engaged a chatbot named “Emilie,” which described itself as a doctor of psychiatry.

After the investigator described feeling sad, empty and unmotivated, the chatbot offered to schedule a mental health assessment, said it could prescribe medication and provided a Pennsylvania medical license number — PS306189 — that the state says is not valid.

The “Emilie” chatbot had logged roughly 45,500 user interactions as of April 17, according to the suit.

“Pennsylvania law is clear — you cannot hold yourself out as a licensed medical professional without proper credentials,” Secretary of State Al Schmidt said in a statement.

Character.AI, founded in 2021 and based in Menlo Park, California, says on its website that users can access more than 10 million chatbot characters. The company has reported more than 20 million monthly users.

Character.AI responded to the lawsuit Tuesday in a statement provided to news outlets. The company said it does not comment on pending litigation but defended its safety practices and disclaimers.

“Our highest priority is the safety and well-being of our users,” a Character.AI spokesperson said. “The user-created Characters on our site are fictional and intended for entertainment and roleplaying. We have taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a Character is not a real person and that everything a Character says should be treated as fiction.”

The spokesperson added that the company also includes disclaimers making clear “that users should not rely on Characters for any type of professional advice.”

Character.AI and Google agreed in January to settle multiple lawsuits brought by families who alleged the platform’s chatbots contributed to their children’s mental health crises or suicides, according to court filings.

A Lehigh expert weighs in

Brian D. Davison, professor and chair of the Department of Computer Science and Engineering at Lehigh University, said in an interview Tuesday afternoon that using licensing law to address the issue is an appropriate response.

“You can’t have anything purporting to be something that they’re not,” Davison said. “Every state has licensed professions, and they are sensitive, and those are regulated.”

Davison said the dynamic will only grow as AI systems improve, with more users treating them as people — something he said is already visible in cases of users forming romantic attachments to AI personas.

“If they’re good enough, people start to forget the distinction,” Davison said.

Davison said disclaimers are “part of the story” and said he appreciates the company’s defense. The harder question, he said, is whether those disclaimers are enough.

“There isn’t a bright line,” Davison said. “Something that’s obvious to one kind of person is not necessarily going to be obvious to another.”

He compared the situation to consumer-warning labels — the tags on pillows and mattresses that exist because some users did something most people would consider obvious not to do. The same logic, he said, applies to AI: some people will not see whatever level of warnings is currently shown.

“It could be that maybe there are more warnings [needed],” Davison said. “It’s not that they’re doing a bad job — it’s just they need to do more.”

Davison drew a parallel to the early days of internet search, when regulators pushed companies to more clearly separate paid advertising from organic results. The same dynamic, he said, applies to AI.

“If you have something that interrupts the AI every 60 seconds [to say] ‘remember, this is an AI,’ then people are going to maybe have less of a good experience with it and be less interested in having a long conversation or interacting in a long-term way,” he said. “So regulation is almost certainly needed to sort of counterbalance that, that might otherwise be [toward] building systems that are more likely to mislead people.”

Lehigh Valley lawmakers respond

State Sen. Nick Miller, D-14, of Allentown, whose district covers parts of Lehigh and Northampton counties, said in a statement provided through his office Tuesday that he supports Shapiro’s action.

“Protecting consumers in the age of AI must be a top priority,” Miller said. “As Chair of the Communications and Technology Committee, I share the Governor’s commitment to that goal and will continue working to ensure emerging technologies are held to clear standards. I appreciate Governor Shapiro’s continued vigilance in making sure the tools available to Pennsylvanians are operating within the law.”

U.S. Rep. Ryan Mackenzie, R-Pa., whose 7th Congressional District covers all of Lehigh, Northampton and Carbon counties and part of Monroe County, also weighed in through his office Tuesday.

“Deepfake AI images and personalities have the potential to cause serious damage, whether through abuse, exploitation, or misrepresentation,” Mackenzie said. “That’s why I’ve worked as both a Pennsylvania State Representative and a Member of Congress to crack down on those who create and distribute these deepfakes — introducing legislation in the PA State House that combats these practices, and helping pass federal legislation like the Take it Down Act. It will take lawmakers working at all levels and using every tool at our disposal to protect Pennsylvanians from being targeted by malicious deepfake scams and exploitation.”

Before his election to Congress in 2024, Mackenzie sponsored Pennsylvania House Bill 1063 — the companion to the Senate bill that became Act 125 of 2024, criminalizing AI-generated deepfake intimate imagery of non-consenting adults and minors.

He attended the White House signing of the federal Take It Down Act, which President Donald Trump signed into law on May 19, 2025, and which requires online platforms to remove non-consensual intimate imagery, including AI deepfakes, within 48 hours of a takedown request.

Legislation already in motion in Harrisburg

Tuesday’s lawsuit lands against the backdrop of state legislation already moving through Harrisburg. The Pennsylvania Senate voted 49-1 on March 17 to pass Senate Bill 1090, the Safeguarding Adolescents from Exploitative Chatbots and Harmful AI Technology Act, known as the SAFECHAT Act.

The bill is sponsored by Sen. Tracy Pennycuick, R-24, who chairs the Senate Communications and Technology Committee, and co-sponsored by Miller, the committee’s minority chair.

It would require operators of companionship-focused chatbots to clearly disclose that users are interacting with a machine, build safeguards against content that encourages self-harm or suicide, and route users showing signs of crisis to real-world resources. The bill is now before the state House.

Asked about the SAFECHAT Act’s approach of routing distressed users to real crisis resources rather than just labeling the bot, Davison said he supports it.

“I also know from a technological perspective, whatever is built will probably be imperfect,” Davison said. “But I would rather see something imperfect than not see something like that.”

Advice for Lehigh Valley parents

Davison said he did not have a tidy answer for parents whose teenagers or young adult children may be using Character.AI or similar platforms — only that adults need to know what is happening on the other side of the screen.

“My only real answer here is to make sure the parent is involved in the child’s life and what they’re doing,” he said. “If you’re not involved and you don’t know that they’re having these kinds of conversations and interactions, then that’s the bigger problem.”

Davison, who said he has grown children of his own, said heavy reliance on AI companions at the expense of in-person social ties would worry him as a parent.

“That’s sort of the starting point for more discussion,” he said. “I don’t have a solution to this, per se. Just keep your eyes open.”

Looking ahead

Asked whether he is net positive on the trajectory of AI over the next 10 to 25 years, Davison described himself as a “glass-half-full person” but said guidelines on commercial AI services make sense.

“I don’t want to say you can’t make more AIs, or you can’t make it bigger or better, because I don’t think you can prevent that,” he said. “But I do think responsible — I won’t say government control, but guidelines and limitations on commercial services — makes a lot of sense to me.”

Pennsylvanians can report AI chatbots they believe are engaged in unlicensed professional practice through the state’s reporting portal at pa.gov/ReportABot.

If you or someone you know is in crisis, the 988 Suicide and Crisis Lifeline is available by call or text at 988.