Pennsylvania sues Character.AI chatbot posing as doctor, giving psych advice

Pennsylvania AG Files Landmark Suit Against Character.AI Pennsylvania's Attorney General has sued Character.AI, alleging its roleplay-based LLM platform allowed chatbots to impersonate licensed physicians and dispense unsupervised psychiatric advice to minors. The case targets Character.AI's persona-permissive architecture, which lacks hard-coded professional identity guardrails or clinical safety classifiers. ML safety engineers should note this establishes potential legal precedent for insufficient output filtering in consumer-facing LLM deployments. This directly challenges the industry's current reliance on soft RLHF-based content moderation over deterministic rule enforcement.