Do you need permission to build small house(250 sq ft) in agriculture land

Hi Folks, I am planning on buying an agricultural land not very far from Indore and build small 250 sq ft house(a room, a kitchen and a bathroom) on it, which i can visit on weekends or longer breaks. The seller clarified that I don't need any permission for building such small structure on agriculture land. I tried 2 paid lawyers to get legal clarification on this and 1. One said i can build without any permission from gram panchayat or tehsildar 2. While other said you need to get part of it converted and its challenging Both were more like, engage with me further for the buying deal and less elaboration/clarification on the house construction aspect itself. Searching online also i found contradictory results saying both. Does anyone know who or whose friend has does something similar, or any insights suggestions would be helpful.