Change ButtonText of a list-trailing on Click in a ListView

As you can see in my code I'm trying to change the text of my button when the string doc.id is in the List pdfIDs. When you have a look at the onPressedAdd function every time the button is clicked, doc.id becomes part of the list. The problem is that the change of the list content doesn't reach the condition which decides the following ButtonText. How can I solve that problem?

Thanks for any kind of help.

class __HomeStateState extends State<_HomeState> {

  List<String> pdfIDs = [];

  Widget _buildList(QuerySnapshot snapshot) {
    return ListView.separated(
      padding: EdgeInsets.all(20),
      separatorBuilder: (context, index) => Container(
        height: 10,
      ),
      itemCount: snapshot.docs.length,
      itemBuilder: (context, index) {
        final doc = snapshot.docs[index];
        return ListTile(
          key: Key(doc.id),
          leading: (doc["ergebnis"] == "Ausstehend")
              ? Icon(Icons.schedule)
              : Icon(Icons.check),
          title: Text(
            doc["nachname"] + ", " + doc["vorname"],
          ),
          subtitle: Text(doc["adresse"] +
              " / " +
              doc["Geburtsdatum"] +
              " / " +
              doc["telefon"]),
          trailing: Wrap(
            spacing: 12,
            children: [
              OutlineButton.icon(
                borderSide: BorderSide(color: Colors.orange),
                onPressed: () {
                  onPressedAdd(doc.id, index);
                },
                icon: Icon(
                  Icons.download_outlined,
                  color: Colors.orange,
                ),
                label: (pdfIDs.contains(doc.id))
                    ? Text(
                        "Add",
                        style: TextStyle(color: Colors.black26),
                      )
                    : Text(
                        "Remove",
                        style: TextStyle(color: Colors.orange),
                      ),
                shape: RoundedRectangleBorder(
                    borderRadius: new BorderRadius.circular(30.0)),
              )
            ],
          ),
        );
      },
    );
  }
}

  void onPressedAdd(String docID, int index) {
    print(pdfIDs.toString());
    if (pdfIDs.contains(docID)) {
      pdfIDs.remove(docID);
    } else {
      pdfIDs.add(docID);
    }
  }