ARI: Hello, I'm Ari. [CLAUDINE: And I'm Claudine] Welcome to Proving the Negative.
ARI: We're a podcast all about exploring the different sides of cybersecurity,
from political to computer science, international relations to mathematics.
Join us as we talk to our friends about the work they do.
CLAUDINE: Today, we're talking about how app design may affect children,
specifically, apps that are targeted towards kids...
and how data collected from those apps could potentially be putting kids at risk.
Ari, is that something you know much about? ARI: I can't say I know much.
It sounds like it's going to tie in to safeguarding (safety/looking after kids).
Does it draw any parallels with your work? CLAUDINE: Not really.
I tend to focus on adults in my research, but this topic is very relevant,
given that online or digital safety for children has been all over the news,
particularly in the UK (legislation like the Online Safety Bill/age appropriate design code).
ANIRUDH: People hear the word cybersecurity
the first thing they think about is hacking and technical things...
Websites, databases, things like that. It's grown way beyond that.
It's not just technical aspects [it's] also about people, what is right and wrong.
It's related to security of computers, to people and however it connects together.
It's as much about people as it is about technical elements.
My name is Anirudh.
I'm a third year student. I'm part of the Human Centred Computing group.
I focus on ethical and responsible app[lication] development for children.
I do this because products that both children and adults make use of,
are designed in such a way to keep us busy,
to keep us occupied and hooked, and to collect as much data as possible.
This can lead to mental health issues: addiction, attention loss or violent behaviour in kids.
Data sharing and loss of privacy can also have consequences.
Once a child grows up and decides to apply for a job,
their CV could go through an automatic profiler,
which might make use of data collected when the individual was a child.
This means that when you're designing services for kids,
there are certain things we have to keep in mind.
We shouldn't try to keep them hooked for as long as possible.
We shouldn't persuade them to disclose unnecessary data.
I'm working with developers to see how we can incentivise apps,
which are age-appropriate and child-friendly.
CLAUDINE: Why are you interested in researching kids, digital resilience,
privacy and responsible design? ANIRUDH: In a way I just rolled into it.
Kids are a vulnerable population and they require different protections from adults,
because adults, they're responsible for their own decisions.
The same cannot be said for kids because they don't make those same decisions.
They might not be fully aware of what privacy is [or] why it's important.
They require extra protections.
I've heard developers say, "you know what, the things we do for children, it..."
"wouldn't be a bad idea to do them for adults as well".
I'm hoping that whatever we do here, we can take that to a wider audience.
Initially I thought I was going to be working [directly] with children.
However, that didn't turn out to be the case. I decided to work with developers.
A lot of the apps on the Google Play Store or the Apple App Store, they're
collecting a lot of data through data trackers and third party libraries.
One of the first questions we had was, "why are developers doing this?",
"are they bad people?", "why are they creating apps which are harmful..."
"... or could potentially be harmful for kids?"
That's how it started out - we interviewed developers and asked them,
"what are your development practices, what do you think about data collection?"
"what do you think you should be doing about this?" That's how it started.
Initially, we didn't really know what to expect.
We thought they would say "we want to earn money, so we're collecting data".
Once we spoke to the developers, we found out they're not bad people at all!
Their motivations are good. They tell us "we don't want to harm children".
We understand they're in their formative years, they're vulnerable and we bear a
certain responsibility when creating apps for children.
However, it's quite challenging to create child-friendly apps.
There are several reasons for that. One of them is analytics.
We need to know what users are doing, how they're behaving.
And this means that developers have to use third party analytics tools.
The most popular ones are from large data controllers such as Google,
meaning that Google has access to all of this data.
It's almost impossible to create apps or services without third party tools,
Developers didn't always know what these are doing in the background.
Sometimes they're collecting data that we're just not aware of.
No one really knows what exactly is happening in the background.
That's the second problem, and the third problem is - at the end of the day,
[developers] have to make money.
How can you compete? Having users pay for the app doesn't work...
first of all, you might not get enough downloads,
so not enough people buy the app for you to make a living.
The only option that's left is targeted advertising,
which means using kids' data to influence behaviours and persuade installations.
Sometimes users want this.
They can't pay for the apps and if [developers] don't make them free,
They're going to leave bad ratings,
forcing developers to remove payment features that make apps free.
These are three challenges we found for developers to create child friendly apps.
The big question for my research is, "what is the solution"?
That's what we're trying to answer. CLAUDINE: What is a tool, or a tool kit?
ANIRUDH: A third party tool or library is software you haven't developed yourself.
Big companies (Google, Facebook, Unity), have third party libraries,
[that] they put out there, saying "we created these, feel free to use them".
Google Analytics is one of them. 'Analytics', in this case means
keeping track of who's coming to your app or website.
Where they are coming from, their gender, how long they use the app...
Very useful, of course. You know exactly what's going on with the service.
A data tracker is not one particular [piece of] software.
It's a generic term for software collecting data and sending it back to a server.
Apps ask for certain permissions... to look into your contacts or record your calls.
Apps like Facebook, they want access to as much as possible.
There is a reason why a lot of these third party libraries are free.
They are making money with it, just not the way you think they are.
They're collecting as much data as they can.
This is used for advertising purposes. This is how Google earns their money.
The moment you click on an ad, they connect advertisers with you.
They get a commission. The reason Google Analytics is free is so they have
data about you, they can show you ads you are more likely to click on.
They'll end up printing more money. The same is true for Facebook.
They want all these permissions, to collect data for exactly the same purposes.
ARI: Why do children and young people need to be resilient?
We put them in environments that we build, and we force them
to interact with each other and adults in these spaces. What are your thoughts?
Digital resilience means children have digital and mental tools,
to cope with the challenges that they're facing.
Browser plug-ins, add-ons for your phone to prevent data sharing...
The mental tools are important as well.
If you find out that your data has been shared or a photo of you has appeared
online, which you didn't want to be online, how do you cope with that?
They are the ones participating in social media, overwhelmed with information.
At the end of the day, they're the ones facing all of that.
That's what digital resilience is about.
Facebook, and a lot of companies, have funders and investors who just want to
make money, because that's very important in this process.
They don't have children's needs in mind. Responsibility to protect whoever is
using [the app] falls on the user. With kids, the responsibility is shifted onto
parents or the caretakers.
We have to do it because there really is no one else - it's not going to get better.
Facebook isn't going to disappear, we're not going to stop developing technology.
If anything, we'll become more connected and children will grow up more and more
in a digital world. The pandemic saw children attending school through the
computer - where before, we taught children to navigate the physical world,
now we have to teach them how to navigate the digital world.
It developed so quickly. 10, 20 years seems like a long time,
but we're not equipped to deal with rapid technological developments.
And now we're trying to catch up, and now we're finding out,
if you're on Facebook, it can actually lead to depression because we're
constantly comparing ourselves with people who seem to have it better.
How do we deal with this? How do we tell children, "OK, you know what?"
"This is not actually the truth". We're playing this game of catch up.
CLAUDINE: There are often assumptions made about levels of competence,
for adults, which isn't necessarily the case.
ANIRUDH: Parents play a big role. They might not know what the dangers are.
They don't have the tools, either. Parents have a responsibility here.
But how do we tell parents about this? How do we educate them?
Should schools have get togethers to discuss these things with parents?
I'm not sure what the answer is. I don't know what these platforms know.
On the one hand, they know exactly what they're doing.
They know exactly how to hook users. Long term consequences need research.
For a child growing up using Facebook, what are the consequences
once that child has become an adult?
How does this person engage with other people?
How does it impact their relationship with their kids or people around them?
These are questions we cannot answer yet, I doubt these platforms know.
These platforms will do research which benefits them.
They'll research:"how can we keep people engaged?"
"How can we improve their experience?", but not so much harm prevention
There's no easy solution to this.
If we tell someone to create an ethical app, how can we make that happen?
There is no strict definition of what an ethical app is, of course,
but let's just say that you don't persuade children to use your app for
hours and hours, and you don't collect unnecessary data.
You're likely to earn less money. That's a big issue.
We don't have a lot of power, actually. If we want to change this ecosystem.
You need cooperation from major players. What Apple has done,
they've added the pop up, or you can opt in to tracking.
So do you agree to this, yes or no? This is good for the end users.
This is not something we could have done ourselves.
We really need Apple or Google or Facebook to cooperate.
We're doing two things. One, we're raising awareness, informing developers:
"Listen, designing for kids is important. These are the reasons, you should do it"
Secondly, a lot of these rules tend to be a little bit abstract.
"Be transparent in what you do". That's great. But what exactly does that mean?
What do you have to do? Does that mean you have to immediately show
your privacy policy the moment people open up your app?
If you have a huge privacy policy, a child is not going to read it.
Raising awareness and developing tools, we hope it's easier to develop for kids.
CLAUDINE: We have developers, social media platforms, advertisers and users.
What role do you think regulation should play in managing this problem?
ANIRUDH: Regulation has started to play a larger role.
A few years back, we saw the GDPR, which changed the landscape a little bit.
In the UK, we saw the age appropriate design codes,
which tell developers and organisations how to develop services for children,
how to make it age-appropriate.
I don't know how much impact they will have. Big organisations are so powerful.
It's hard to bypass them. A few years ago, Shoshana Zuboff wrote a book
called "The Age of Surveillance Capitalism". She describes how
organisations circumvent the role of governments and regulation.
One of the things they do is to make their practises the new normal,
The example she gave was Google Maps.
What Google started doing was going around and photographing everything
people's houses with cars, and suddenly it became the new normal.
We were so used to having these maps available that no one questioned them.
When there was pushback, they made minor changes, not really changes at all.
Then we forgot about it. Regulation can sometimes be a little bit slow.
By the time it tackles issues, we might have gotten used to [a] new way of life.
CLAUDINE: In terms of children and their use of apps and social
media, the collection of their data could potentially harm them in the long term.
What did you mean by that? ANIRUDH: There are several different harms.
One is mental health and well-being, for example, addiction.
If you're scrolling Facebook or if you're playing an online game, they're
designed in such a way that they keep you busy for as long as possible.
And addiction can lead to all types of problems - loss of sleep,
increased violence, etc. Depression anxiety when using social media.
The second category which is important is privacy. The harms seem very far away.
They're not immediately visible.
But these relate to decreased opportunities in life,
and risks associated with data theft, fraud, identity theft.
Privacy is, of course, a huge part of security. It's all related.
Some of the harms related to loss of privacy are not immediately clear.
Google collects my data, but I haven't felt anything. Nothing has happened.
So what exactly is the issue? There are harms which can arise from this.
Data theft - Google or Facebook can be hacked,
suddenly your data is in the hands of someone else, they can use it for fraud.
They might get a hold of your bank details, they could impersonate you.
More interestingly, are harms associated with long term opportunities.
Facebook and Google might have data from you which they've collected over
the course of, let's say, 20, 30 years.
They might be sharing this with third parties.
Insurance rates might be higher because of previous behaviours.
It could simply be how you used your mobile device - lots of hand movement,
you're less careful. Your insurance rates are higher.
Is that fair? How you behaved 20 years ago might be very different to now.
This is an example of how loss of privacy and data sharing happening when you
are a child can impact you when you're a lot older.
There have been examples where CVs have been scanned by an algorithm
and [the algorithm] profiled against women, for example.
Data collected when you were a kid from a toy or website, combined with
your CV, suddenly you got rejected.
That's an example of how data sharing can impact you negatively.
CLAUDINE: If you had unlimited time, money, personnel and resources,
how would you tackle some of the challenges that you face?
ANIRUDH: I'm interested in mindfulness.
I'd be curious to find out if digital resilience could be added as a subject.
Kids could be taught about the issues we've discussed, so they're more
equipped to handle it later on in their lives.
This would be interesting for researchers and beneficial in the long term.
This might be one of the things I would allocate those funds to.
ARI: What have you and your colleagues learned from kids and young people?
ANIRUDH: What we have learnt from kids is what their needs are.
We keep talking about making [apps] ethical, child friendly, age appropriate.
But what I've not discussed is what that means. We can learn from kids.
We can ask "what is important to you?", "what are your needs?"
We can think about how this feeds into what developers are creating.
We can keep that in mind - for example, don't make a pop-up notification at
eight o'clock in the morning because they should be focused on school.
What we learn from them we can use to inform developers.
CLAUDINE: What is next for you? ANIRUDH: Developing for kids is critical,
beyond the UK. Right now, we're focused on the UK market. Next,
we are hoping to develop a collection of tools for developers which makes
age-appropriate and child-friendly design for kids easier.
For example, a video course, design templates, code libraries...
[We'll ask devs] "What exactly will be useful for you, what will you use?"
Then prototype and develop [the tools] and release them to the wider audience.
We'll establish a community around that which we can take internationally.
CLAUDINE: Where can people keep up with the work that you're doing?
ANIRUDH: I don't have a huge online presence. I've got an Oxford page...
CLAUDINE: If people are interested in learning more about the topic, what
kind of resources would you suggest that they look at?
ANIRUDH: General news sources, Twitter influencers can be quite useful.
CLAUDINE: That was our interview with Anirudh.
Join us next week for another fascinating conversation.
In the meantime, you can tweet at us @HelloPTNPod.
You can subscribe on Apple Podcasts or wherever you listen to podcasts.
The title there is PTNPod. See you next week. ARI: Bye!
CLAUDINE: This has been a podcast from the Centre for Doctoral Training in Cybersecurity, University of Oxford.
Funded by the Engineering and Physical Sciences Research Council.