SCSS - Hover on button with SVG has an unintended delay effect

I have a Button that contains an SVG icon element. When hovering over it, I want to recolor both, the SVG icon and the text. My SCSS looks like so:

&.btn-highlight-green {
    color: $color-green;
    background-color: white;
    border: 1px solid $color-green;

    > svg {
        color: $color-green;
    }

    &:hover {
        color: $color-green-dark;
        border: 1px solid $color-green;
        background-color: $color-green-light;

        > svg {
           color: $color-green-dark;
        }
    }
}

It does what I intend, but when I hover, first the svg icon is recolored, and then with a delay of <1 sec the text is recolored. How can I avoid that effect? especially when only quickly hovering over it it basically only recolors the svg.