I updated my canonical URL and the rankings dropped! Should I change back?

This is very important to me and I would appreciate any help\~. Here's the thing, My site is a B2C site with many products, so we opted for a pagination setup. Last month, I set all paginated canonical URLs to page 1 (URLs without page numbers), and 18 days after the change, I noticed that my site's rankings had all dropped. Should I go back to the original canonical settings? (Originally, each page was canonical to its URL)