Leaders of Telugu Desam Party (TDP) alleged that Naidu and his family members came to know about this after their return from a foreign trip on Tuesday.
The security of TDP chief N. Chandrababu Naidu's son Nara Lokesh, who earlier had 'Z' category cover, has been now scaled down to 'Y'. He will now get 2+2 security guards against 5+5 earlier, TDP sources said.
The security of other family members has been completely withdrawn. The TDP alleged that it was done without informing them.
Lokesh, who was minister in his father's cabinet, was defeated in April 11 elections which saw TDP losing to power YSR Congress Party (YSRCP).
Naidu, his wife Bhuvaneswari, son Lokesh, daughter-in-law Brahmani and grandson Devaansh landed in Hyderabad after their holiday trip to Europe.
Naidu, who had escaped assassination attempt by Maoists in 2003, has since been enjoying 'Z plus' security, the highest security cover enjoyed by VIPs in the country.
The opposition party has termed the latest move by YSR Congress Party (YSRCP) government as another act of vendetta politics.
It came a day after Chief Minister Y. S. Jagan Mohan Reddy announced that a building built as an extension to Naidu's residence on the banks of Krishna river in Amaravati will be demolished on Wednesday. He said the building called 'Praja Vedika' was built in violation of all laws and rules.
At the meeting of district collectors and SPs at 'Praja Vedika', Jagan Reddy said Naidu himself was living in an illegal building. He announced that the drive against illegal structures would begin Wednesday with the demolition of 'Praja Vedika'.
( With inputs from IANS )