Colección SciELO Chile

Departamento Gestión de Conocimiento, Monitoreo y Prospección
Consultas o comentarios: productividad@anid.cl
Búsqueda Publicación
Búsqueda por Tema Título, Abstract y Keywords



Differentially Private Algorithms for the Stochastic Saddle Point Problem with Optimal Rates for the Strong Gap
Indexado
WoS WOS:001222719102019
Scopus SCOPUS_ID:85171578631
DOI
Año 2023
Tipo proceedings paper

Citas Totales

Autores Afiliación Chile

Instituciones Chile

% Participación
Internacional

Autores
Afiliación Extranjera

Instituciones
Extranjeras


Abstract



We show that convex-concave Lipschitz stochastic saddle point problems (also known as stochastic minimax optimization) can be solved under the constraint of (ϵ, δ)-differential privacy with strong (primal-dual) gap rate of Õ(√1n + √nϵd), where n is the dataset size and d is the dimension of the problem. This rate is nearly optimal, based on existing lower bounds in differentially private stochastic convex optimization. Specifically, we prove a tight upper bound on the strong gap via novel implementation and analysis of the recursive regularization technique repurposed for saddle point problems. We show that this rate can be attained with O(min {n2√ϵd1.5 , n3/2}) gradient complexity, and Õ(n) gradient complexity if the loss function is smooth. As a byproduct of our method, we develop a general algorithm that, given a black-box access to a subroutine satisfying a certain α primal-dual accuracy guarantee with respect to the empirical objective, gives a solution to the stochastic saddle point problem with a strong gap of Õ(α+ √1n). We show that this α-accuracy condition is satisfied by standard algorithms for the empirical saddle point problem such as the proximal point method and the stochastic gradient descent ascent algorithm. Finally, to emphasize the importance of the strong gap as a convergence criterion compared to the weaker notion of primal-dual gap, commonly known as the weak gap, we show that even for simple problems it is possible for an algorithm to have zero weak gap and suffer from Ω(1) strong gap. We also show that there exists a fundamental tradeoff between stability and accuracy. Specifically, we show that any ∆-stable algorithm has empirical gap Ω(∆1n), and that this bound is tight. This result also holds also more specifically for empirical risk minimization problems and may be of independent interest.

Revista



Revista ISSN
2640-3498

Disciplinas de Investigación



WOS
Sin Disciplinas
Scopus
Sin Disciplinas
SciELO
Sin Disciplinas

Muestra la distribución de disciplinas para esta publicación.

Publicaciones WoS (Ediciones: ISSHP, ISTP, AHCI, SSCI, SCI), Scopus, SciELO Chile.

Colaboración Institucional



Muestra la distribución de colaboración, tanto nacional como extranjera, generada en esta publicación.


Autores - Afiliación



Ord. Autor Género Institución - País
1 Bassily, Raef - The Ohio State University - Estados Unidos
College of Engineering - Estados Unidos
OHIO STATE UNIV - Estados Unidos
2 Guzman, Cristobal Hombre Facultad de Matemáticas - Chile
Pontificia Universidad Católica de Chile - Chile
3 Menart, Michael Hombre The Ohio State University - Estados Unidos
College of Engineering - Estados Unidos
OHIO STATE UNIV - Estados Unidos
4 Neu, G -
5 Rosasco, L -

Muestra la afiliación y género (detectado) para los co-autores de la publicación.

Financiamiento



Fuente
FONDECYT
National Science Foundation
Fondo Nacional de Desarrollo Científico y Tecnológico
NSF
NSF CAREER Award
Institut national de recherche en informatique et en automatique (INRIA)
Agencia Nacional de Investigación y Desarrollo
ANID Anillo
Basal ANID
National Center for Artificial Intelligence CENIA
INRIA Associate Teams project

Muestra la fuente de financiamiento declarada en la publicación.

Agradecimientos



Agradecimiento
RB’s and MM’s research is supported by NSF CAREER Award 2144532 and NSF Award AF-1908281. CG’s research was partially supported by INRIA Associate Teams project, FONDECYT 1210362 grant, ANID Anillo ACT210005 grant, and National Center for Artificial Intelligence CENIA FB210017, Basal ANID.
RB's and MM's research is supported by NSF CAREER Award 2144532 and NSF Award AF-1908281. CG's research was partially supported by INRIA Associate Teams project, FONDECYT 1210362 grant, ANID Anillo ACT210005 grant, and National Center for Artificial Intelligence CENIA FB210017, Basal ANID.

Muestra la fuente de financiamiento declarada en la publicación.