Statistical Inference Without Excess Data Using Only Stochastic Gradients: Volume 1

Details

• Alternative Title:
  Solving Perception Challenges for Autonomous Vehicles Using SGD [Project Title, from cover]
• Creators:
  Li, Tianyang ; Liu, Liu ; Kyrillidis, Anastasios ; Caramanis, Constantine
• Corporate Creators:
  University of Texas at Austin. Data-Supported Transportation Operations & Planning Center (D-STOP)
• Corporate Contributors:
  United States. Department of Transportation. University Transportation Centers (UTC) Program
• Subject/TRT Terms:
  Algorithms Autonomous Vehicles Convex Optimization Frequentist Inference High Dimensional Statistics Linear Regression Analysis M-estimation Machine Learning Statistical Inference Time Series Time Series Analysis

  More +
• Publication/ Report Number:
  D-STOP/2020/159
• Resource Type:
  Tech Report
• Geographical Coverage:
  United States
• Corporate Publisher:
  University of Texas at Austin. Data-Supported Transportation Operations & Planning Center (D-STOP)
• Abstract:
  We present a novel statistical inference framework for convex empirical risk minimization, using approximate stochastic Newton steps. The proposed algorithm is based on the notion of finite differences and allows the approximation of a Hessian-vector product from first-order information. In theory, our method efficiently computes the statistical error covariance in M-estimation, both for unregularized convex learning problems and high-dimensional LASSO regression, without using exact second order information, or resampling the entire data set. We also present a stochastic gradient sampling scheme for statistical inference in non-i.i.d. time series analysis, where we sample contiguous blocks of indices. In practice, we demonstrate the effectiveness of our framework on large-scale machine learning problems, that go even beyond convexity: as a highlight, our work can be used to detect certain adversarial attacks on neural networks.
• Format:
  PDF
• Funding:
  DTRT13-G-UTC58
• Collection(s):
  University Transportation Centers US Transportation Collection Autonomous Vehicles
• Main Document Checksum:
  urn:sha-512:7252aa1b492d92dac557ab1665fd8a2014475a432f2213b4208ccae62ee920725b2e2d1b52d3db6eeeb258f761cb6b101e6004298910136eb2a53da7f5305115
• Download URL:
  https://rosap.ntl.bts.gov/view/dot/55860/dot_55860_DS1.pdf
• File Type:
  [PDF - 926.37 KB ]