So I just changed the oil(which it turns out measuring the dirty oil was a quart low) and while stopped at a light noticed smoke coming from the bonnet, pulled over, popped the hood and saw smoke coming from the O2 Sensor and a sheen of oil down that heat shield.

I've always changed my oil since I've gotten the car, but i took it to the dealership for another issue, and now I'm a quart lower now since taking it to the dealer.