This is an exception to the general electronegativity trend. It can be explained by looking at the electron configurations of both elements.
Be:[He]2s2
B:[He]2s22p1
When you remove an electron from beryllium, you are taking away an electron from the 2s orbital. When you remove an electron from boron, you are taking an electron from the 2p orbital. The 2p electrons have more energy than the 2s, so it is easier to remove them as they can more strongly resist the effective nuclear charge of the nucleus.
