Apparatus and method for transmitting and receiving data in a multiuser mimo system

A method for transmitting and receiving data in a multiuser Multiple-Input Multiple-Output (MIMO) system including a base station connected to a plurality of user equipments. In the method, each of the user equipments separately calculates channel capacity information using a linear reception scheme and a nonlinear reception scheme, and feeds back the calculated channel capacity information to the base station. The base station separately calculates system capacities using at least two different antenna scheduling schemes based on the channel capacity information fed back from the user equipments, and allocates antennas to the user equipments using a selected antenna scheduling scheme having higher system capacity out of the two antenna scheduling schemes.