Firebase Web Authentication - what about server side?

I'd like to authenticate users using Firebase and Firebase UI. I have just started out with it. I was thinking to use it as all tutorials suggest, and that is checking for a valid user in the promise, providing the developers a way to get that user's email, uid and the provider (maybe a picture too).

But then I thought, what about my server side? couldn't someone just copy the previously logged in user's email, and by using devtools just populate the fields as they see fit, and then just start using the web app?

Isn't there a way for my server to query Firebase under the hood so that I can be sure that user is still logged in?

Would that accessToken might be useful for that? Is there a library that helps contacting Firebase servers?

{"displayName”:”someone”
"email”:”some@one.com",
"emailVerified":false,
"phoneNumber":null,
"photoURL":"http://ift.tt/2DoKRAg?,
"uid":"6yC3n39SsnOeFNZ6pJ0Yaw1vF2e3",
"accessToken”:”hjeher38738743j4k34.....", // <-- IMMEDIATE SUSPECT
"providerData":[
{
"uid":"987239872349872",
"displayName":"Some User",
"photoURL":"https://scontent.xx.fbcdn.net/v/t1.0-1/p100x100/83487843_3453453453453$%_32487234.jpg",
"email":"some@mail.com",
"phoneNumber":null,
"providerId":"facebook.com"
}
]