Just as a first point, I've never been 100% sure if I needed to use pull-up resistors with I2C on an Arduino, as most of the time I haven't needed them. This time though they seemed to make a difference. I used 4.7kohm resistors to pull up SDA and SCL.
Here's the code:
A bunch of credit goes to the post on Hack A Day as well as the post on Semyon Tushev's blog (links below). I should note that I didn't really spend much time with the registers as I only wanted to get 16.2kHz out of it (for now) so I won't be very knowledgeable about that stuff.
ReferencesParts: 133MHz-16.2kHz programmable oscillator (DS1077)
Measuring frequency with Arduino
DS1077 Programmable Oscillator Breakout - 16.2kHz to 133MHz
Arduino Forum: Need for I2C pullup resistors