We don't worship the Pope, nor any priest but Christ. Christ is the only man so we can worship as he is God. Papal authority is only so because it derives from the authority of Christ, it is never separate and cannot exist without Christ.