I renamed my scripts folder and the severity of the frame drops seemed to decrease. I didn't get them as often and they only dropped to 25 or 30 fps. Then I renamed my mods folder as well and the severity seemed to go down even more, only dropping to 45 fps or so. But they were still occurring.
I guess this means either an issue with my game or with my system?