**Revised Answer**

There were some errors in my previous answer.

It is not quite true that bouncing can happen **only** when particle A reaches the top of the hoop. I think what you mean is that bouncing **occurs for the lowest value of $v_0$** when particle A is at the top of the hoop. At faster speeds it can occur before A reaches the top.

Your calculation so far is correct. Bouncing starts when the upward centrifugal force caused by the rotation of particle A around its average position exceeds the combined weight of the hoop and particle A. When the velocity of the centre C of the hoop is $v$ the velocity of A relative to C is also $v$. Therefore bouncing starts when $$m\frac{v^2}{R} \gt 2mg$$ $$v^2 \gt 2gR$$ Bouncing can occur before A reaches its highest position if the **vertical upward component** of the centrifugal force exceeds $2mg$.

As you realise, we have to use conservation of energy to find $v$, the seed of the hoop when A reaches its highest position.

When A is at its lowest point it is stationary and has no KE. The COM of the hoop is moving with velocity $v_0$ so the hoop has translational KE $\frac12 mv_0^2$. The hoop also has rotational KE. In the COM frame every point on the hoop is moving at speed $v_0$ so the hoop has rotational KE of $\frac12 mv_0^2$. The total energy when A is at its lowest point is therefore $mv_0^2$.

When A reaches its highest point the centre of the hoop has some unknown speed $v$ and particle A has speed $2v$. The translational KE of the hoop and particle A are $\frac12 mv^2$ and $2mv^2$ respectively. The hoop also has rotational KE of $\frac12 mv^2$ as calculated above. The total KE in this position is $3mv^2$. The gravitational PE of A has increased by $2mgR$.

Total mechanical energy is the same in both positions. Applying the inequality above we get $$mv_0^2=3mv^2+2mgR$$ $$v_0^2-2gR = 3v^2 \gt 3(2gR) = 6gR$$ $$v_0^2 \gt 8gR$$

This answer agrees with solutions on YouTube and Brilliant.org.