AI Ethics and Children's Photos: Our Three Hard Limits

An AI model that turns a child’s photo into an illustration within seconds raises questions that go beyond privacy alone. Consent, impersonation of real people, and exploitation of vulnerable users are three separate ethical categories, and children are increasingly treated as their own, stricter category in regulation. The EU AI Act (Regulation (EU) 2024/1689) explicitly prohibits AI systems that exploit the vulnerabilities of a specific group - children among them - to influence behaviour in ways that cause harm. UNICEF published a dedicated policy guidance on AI and children in 2021, with a core point that children are not simply “small adults” for AI design purposes: their rights, vulnerabilities and stage of development call for their own considerations, separate from what counts as adequate protection for adults.

For a service that generates illustrations from an uploaded photo of a child, that is not an abstract debate. At MyOwnChildbook, we draw three hard limits that stand apart from what is technically possible.

Limit one: no photo without a parent’s consent

The first limit is the most obvious, and at the same time the hardest to enforce technically: only the parent or guardian of the child may upload the photo, confirmed explicitly at the point of upload. No system can verify with certainty who is actually in a photo or who that person is to the uploader. What is achievable is limiting the risk after the upload: no photo is stored permanently, shared between accounts, or reused for a different book than the one it was uploaded for. Once the illustrations are generated and approved, the photo is automatically removed from our servers.

That is a deliberate choice: consent itself cannot be verified, but the risk of misuse after upload can be minimised. You can read more about the technical side of that promise in our article on what actually happens to an uploaded photo.

Limit two: no impersonation of real, recognisable people

The second limit sits closer to the technology itself. The system is not designed to reproduce a specific, named public figure, nor to generate a different child than the one uploaded based on a text description. That is the same underlying logic behind why our own style options never name a specific illustrator or studio: safety classifiers in models such as gpt-image-2 react sharply to the combination of a name and a child context, and rightly so. A system that has learned to be cautious with “name plus child” is exactly the kind of guardrail you want for content made with and about children.

Edwin, the data engineer behind the pipeline: “We deliberately chose to describe styles rather than name them. That is not only a technical workaround to avoid refusals, it is also the right substantive choice: you do not want to build a system that makes it easy to tie a child’s face to the work or identity of a specific, real third party.”

Limit three: no exploitative patterns

The third limit concerns the design of the service itself, not the illustration. No advertising aimed directly at children, no artificial time pressure or fake scarcity in the order flow, and no profiling of children for advertising purposes. The person filling in the form and paying is the parent, and the service is built around that: the child is the subject of the book, never the target of marketing.

That sounds obvious, but the EU AI Act explicitly names this kind of “dark pattern” aimed at vulnerable groups as exactly the type of manipulation the legislation is meant to prevent. For a service aimed directly at families with young children, stating that limit out loud rather than assuming it silently is not a redundant exercise.

When this is still not enough certainty

For some parents, no guarantee is enough: they have a principled objection to entrusting any photo of their child to an AI system, however the safeguards are set up. That is a legitimate position, not distrust that needs correcting. For that situation, a book that personalises through a description rather than a photo upload - the way Wonderbly does, using name, hair colour and skin tone - is the more honest choice. Our own comparison between Wonderbly and MyOwnChildbook sets that difference out side by side.

Edwin’s side of the story

Edwin builds the pipeline and is himself a father of two. “As a data engineer, I can explain how the deletion and anonymisation work technically. As a father, I ask myself a simpler question: would I upload a photo of my own kids to a service I had not built myself? That question keeps me sharper than a compliance checklist. We describe the character-consistency side - how we keep your child’s face recognisable from the first page to the last - in a separate article on that technical piece.”

Ethics in AI and children’s content is not a box you tick once and consider solved. It is an ongoing balance between what is technically possible, what the law requires, and what you would want for your own child as a parent. That last test is, in the end, the strictest of the three.

👉 See how a personalised children’s book from MyOwnChildbook works