๐Ÿ“š Study 119

[Paper Review] Compact3D: Smaller and Faster Gaussian Splatting with Vector Quantization

0. Abstract1. ๋งŽ์€ Gaussian๋“ค์ด similar parameters๋ฅผ ๊ณต์œ ํ•œ๋‹ค๋Š” ์ ์„ ๋ฐœ๊ฒฌํ–ˆ๋‹ค2. ๋”ฐ๋ผ์„œ, Gaussian parameters๋ฅผ quantizeํ•˜๊ธฐ ์œ„ํ•ด K-means ๊ธฐ๋ฐ˜์˜ vector quantization ๋ฐฉ๋ฒ•๋ก ์„ ์ œ์‹œํ•œ๋‹ค.์ด๋•Œ ๊ฐ๊ฐ์˜ Gaussian์˜ ์ฝ”๋“œ index์™€ ํ•จ๊ป˜ codebook์„ ์ €์žฅํ•œ๋‹ค.์ถ”๊ฐ€๋กœ ์ธ๋ฑ์Šค๋Š” ์ •๋ ฌํ•œ ํ›„ run-length encoding๊ณผ ๋น„์Šทํ•œ ๋ฐฉ๋ฒ•๋ก ์„ ์‚ฌ์šฉํ•จ์œผ๋กœ์จ ์••์ถ•ํ•œ๋‹ค.3. Gaussian ์ˆ˜๋ฅผ ์ค„์ด๊ธฐ ์œ„ํ•ด zero opacity (invisible Gaussian)์„ ์žฅ๋ คํ•˜๋Š” regularizer๋„ ์ œ์•ˆํ•œ๋‹ค.์ด ๋ฐฉ๋ฒ•์€ ๋ชจ๋ธ์„ ์••์ถ•ํ•˜๊ณ  rendering ์†๋„๋ฅผ ๋น ๋ฅด๊ฒŒ ํ•˜๋Š”๋ฐ ํšจ๊ณผ์ ์ด๋‹ค.4. ๊ฒฐ๋ก ์ ์œผ๋กœ ๊ธฐ์กด 3DGS์— ๋น„ํ•ด ์„ฑ๋Šฅ์€ ์•ฝ๊ฐ„..

K-means Clustering์—์„œ centroid์™€ assignment ์—…๋ฐ์ดํŠธํ•˜๋Š” ๊ฒƒ ์ค‘ ๋ญ๊ฐ€ ๋” ์˜ค๋ž˜ ๊ฑธ๋ฆด๊นŒ?

K-means has two steps: updating centroids given assignments, and updating assignments given centroids. We note that the latter is more expensive while the former is a simple averaging. Hence, we update the centroids after each iteration and update the assignments once every t iterations.์ถœ์ฒ˜) Compact3D: Smaller and Faster Gaussian Splatting with Vector Quantization ๋…ผ๋ฌธ์„ ์ฝ๋‹ค๊ฐ€ ์™œ ๊ทธ๋ ‡์ง€? ์ดํ•ด๊ฐ€ ์ž˜ ์•ˆ๋˜๋Š” ๋ถ€๋ถ„์ด ์žˆ์–ด์„œ..

[๋”ฅ๋Ÿฌ๋‹๊ณผ ์„ค๊ณ„] GAN

# GAN  VAE๊ฐ™์€ ๊ฒฝ์šฐ๋Š” ๊ฒฝํ—˜์ ์ด ์•„๋‹Œ ์ˆ˜ํ•™์ ์œผ๋กœ ์ ‘๊ทผํ•˜๋‹ค๋ณด๋‹ˆ๊นŒ ๋„ˆ๋ฌด ๋ณต์žกํ•ด์„œ,๊ทธ๋ƒฅ samplingํ–ˆ์„ ๋•Œ ๊ทธ๋Ÿด ๋“ฏํ•œ ๊ฒฐ๊ณผ๊ฐ€ ๋‚˜์˜ค๊ฒŒ๋งŒ ๋งŒ๋“ค๋ฉด ์•ˆ๋ ๊นŒ?explicit density functionํ•˜์ง€ ๋ง๊ณ  game theory๋ฅผ ์‚ฌ์šฉํ•ด๋ณด์ž! game theory: 2-player game์ด๋ž€?A๊ฐ€ B์˜ ๊ฒฐ๊ณผ๋ฅผ ๋ณด๊ณ  ์ตœ์ ํ™”ํ•˜๊ณ ,B๋Š” A๊ฐ€ ์ตœ์ ํ™”ํ•œ ๊ฒƒ์„ ๋ณด๊ณ  ๋˜ ์ตœ์ ํ™”ํ•œ๋‹ค.์ด ๊ณผ์ •์„ ๋ฐ˜๋ณตํ•˜๋‹ค๋ณด๋ฉด, A์™€ B๊ฐ€ ๋‘˜๋‹ค ์ด์ต์„ ์–ป์„ ์ˆ˜ ์—†๋Š” ๋‹จ๊ณ„์— ์ด๋ฅธ๋‹ค.  decoder๊ฐ€ generator, ๊ทธ๋ฆฌ๊ณ  encoder๊ฐ€ discriminator์ด ๋˜๋Š” ๊ตฌ์กฐ๋‹ค.์ฆ‰, ๋จผ์ € generator๊ฐ€ fake image๋ฅผ ๋งŒ๋“ค๊ณ discriminator๋Š” real image์™€ fake image๋ฅผ ๋ฐ›๊ณ  ๊ฐ๊ฐ์ด real(1)์ธ์ง€ ..

๐Ÿ“š Study/AI 2024.07.11

[๋”ฅ๋Ÿฌ๋‹๊ณผ ์„ค๊ณ„] VAE(Variational AutoEncoder)

๋ณธ ๊ธ€์€ ์•„๋ž˜ ์˜์ƒ์„ ๊ธฐ๋ฐ˜์œผ๋กœ ์ž‘์„ฑํ•˜์˜€์Šต๋‹ˆ๋‹ค.https://www.youtube.com/watch?v=GbCAwVVKaHY&list=PLQASD18hjBgyLqK3PgXZSp5FHmME7elWS&index=10  # Variational Autoencoders(VAE) AE ๊ฐ™์€ ๊ฒฝ์šฐ์—๋Š” encoder๊ฐ€ ์ค‘์š”ํ•œ ๋ฐ˜๋ฉด, VAE๋Š” decoder๊ฐ€ ๋” ์ค‘์š”ํ•˜๋‹ค.์ฆ‰, AE๋Š” ์ฐจ์›์„ ์ถ•์†Œํ•˜๋Š” ๊ฒŒ ์ค‘์š”ํ•˜๊ณ , VAE๋Š” ์ƒˆ๋กœ์šด ๋ฐ์ดํ„ฐ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ๊ฒƒ์ด ์ค‘์š”ํ•˜๋‹ค. encoder๊ฐ™์€ ๊ฒฝ์šฐ์—๋Š” ๋ฐ”๋กœ $z$๋ฅผ ๊ตฌํ•˜๋Š”๊ฒŒ ์•„๋‹ˆ๋ผํ‰๊ท  $\mu$์™€ ๋ถ„์‚ฐ $\sigma$๋ฅผ ๋ฝ‘์•„๋‚ธ ํ›„ ์ƒ˜ํ”Œ๋งํ•ด์„œ $z$๋ฅผ ๊ตฌํ•œ๋‹ค.๊ทธ๋ฆฌ๊ณ  ์—ฌ๊ธฐ์„œ samplingํ•˜๋Š” ๊ณผ์ •์—์„œ Reparameterization Trick $e$์„ ์‚ฌ์šฉํ•ด์•ผ backpropagat..

๐Ÿ“š Study/AI 2024.07.10

[๋”ฅ๋Ÿฌ๋‹๊ณผ ์„ค๊ณ„] Autoencoder & Anomaly Detection

๋ณธ ๊ธ€์€ ๋‹ค์Œ ๊ฐ•์˜๋ฅผ ๋“ฃ๊ณ  ์ •๋ฆฌํ•˜์˜€์Šต๋‹ˆ๋‹ค.https://www.youtube.com/watch?v=9mf4maQU7UY&list=PLQASD18hjBgyLqK3PgXZSp5FHmME7elWS&index=7    # Autoencoder Autoencoder์€ input๊ณผ output์ด ๋™์ผํ•œ ๋”ฅ๋Ÿฌ๋‹ ๋ชจ๋ธ์ด๊ณ ,Encoding์€ ๊ณ ์ฐจ์›์˜ ๋ฐ์ดํ„ฐ๋ฅผ ์ €์ฐจ์›์œผ๋กœ ์••์ถ•ํ•˜๋Š” ๊ณผ์ •์„ ์˜๋ฏธํ•œ๋‹ค. (ex. ์œ„์˜ ๊ทธ๋ฆผ, 100์ฐจ์› -> 2์ฐจ์›)๊ทธ๋ฆฌ๊ณ  ์••์ถ•ํ•œ ๊ฒƒ์„ ๋‹ค์‹œ ์›๋ž˜์˜ input ํ˜•ํƒœ์˜ ํฌ๊ธฐ๋กœ ๋ณต์›ํ•˜๋Š” Decoding์ด ์žˆ๋‹ค. ์••์ถ•ํ•œ ๋ถ€๋ถ„์„ ์˜๋ฏธํ•˜๋Š” $z$๋Š” code, latent variable, feature, hidden representation ๋“ฑ ๋‹ค์–‘ํ•˜๊ฒŒ ๋ถ€๋ฅธ๋‹ค. ๋ถ„๋ช… ๋ณธ ์ฐจ์›์„ ์ค„์—ฌ latent varia..

๐Ÿ“š Study/AI 2024.07.10

[๋”ฅ๋Ÿฌ๋‹๊ณผ ์„ค๊ณ„] Unsupervised Learning ๋น„์ง€๋„ํ•™์Šต๊ธฐ์ดˆ

๋ณธ ๊ฒŒ์‹œ๊ธ€์€ ๋‹ค์Œ ๊ฐ•์˜๋ฅผ ๋“ฃ๊ณ  ์ •๋ฆฌํ•œ ๋‚ด์šฉ์ž…๋‹ˆ๋‹ค.https://www.youtube.com/watch?v=V9HcvXliJmw&list=PLQASD18hjBgyLqK3PgXZSp5FHmME7elWS&index=6     # Basic Probability  supervised learning๊ณผ ๋‹ค๋ฅด๊ฒŒ unsupervised learning์—์„œ๋Š”'ํ™•๋ฅ '์— ๋Œ€ํ•œ ๊ฐœ๋…์ด ๋งŽ์ด ๋‚˜์˜ค๊ธฐ ๋•Œ๋ฌธ์— ์ด๋ฅผ ๋‹ค์‹œ ๊ณต๋ถ€ํ•˜๊ณ  ๋„˜์–ด๊ฐˆ ํ•„์š”๊ฐ€ ์žˆ๋‹ค.-(์กฐ๊ฑด๋ถ€ ํ™•๋ฅ  ex.)$ p(x1,x2,x3) = p(x1|x2, x3) * p(x2|x3) * p(x3) $-(์ „์ฒด ํ™•๋ฅ ์˜ ๋ฒ•์น™)$ p(y) = \sum_{x} p(x,y) = \sum_{x} p(y|x)p(x) $ ์ด๊ฑธ ์—ฐ์†์ ์ธ data์— ๋Œ€ํ•ด ์ž‘์—…์„ ํ–ˆ์„ ๋•Œ๊ฐ€ (Marginal..

๐Ÿ“š Study/AI 2024.07.10

[cs231n] Variational Autoencoders (VAE)

์ด์ „ PixelCNN ๊ฐ™์€ ๊ฒฝ์šฐ์—๋Š”, ํ™•๋ฅ  ๋ชจ๋ธ์ด ๊ณ„์‚ฐ ๊ฐ€๋Šฅํ•œ ํ•จ์ˆ˜์˜€๋Š”๋ฐ,VAE(Variational Autoencoders)๋Š” ํ™•๋ฅ  ๋ชจ๋ธ์ด ๊ณ„์‚ฐ ๋ถˆ๊ฐ€๋Šฅํ•œ ํ•จ์ˆ˜๋กœ ์ •์˜๊ฐ€ ๋œ๋‹ค.๋”ฐ๋ผ์„œ, Lower bound(ํ•˜ํ•œ์„ )์„ ๊ตฌํ•ด์„œ ๊ณ„์‚ฐ ๊ฐ€๋Šฅํ•œ ํ˜•ํƒœ๋กœ ๋งŒ๋“ค์–ด์ฃผ๋Š”๊ฒŒ ๋ชฉ์ ์ด๋‹ค. VAE์— ๋Œ€ํ•ด ๋ฐ”๋กœ ๋“ค์–ด๊ฐ€๊ธฐ ์ „์—,Autoencoder์˜ ๊ณผ์ •์ธ Encoder์™€ Decoder์— ๋Œ€ํ•ด ์•Œ์•„๋ณด์ž.   Autoencoder์ด๋ž€ input data $x$๋กœ๋ถ€ํ„ฐ ๋” ๋‚ฎ์€ ์ฐจ์›์˜ feature $z$๋ฅผ ํ•™์Šตํ•˜๋Š” ๋ฐฉ๋ฒ•์ด๋‹ค.$z$๊ฐ€ $x$๋ณด๋‹ค ์ฐจ์›์ด ๋‚ฎ์€ ์ด์œ ๋Š”, ๊ธฐ์กด์˜ input ์ค‘์—์„œ 'ํ•ต์‹ฌ ์ •๋ณด'๋งŒ์„ ๊ฐ–๊ณ  ์žˆ๊ธฐ ๋•Œ๋ฌธ์ด๋‹ค.์ฆ‰, encoder๋ฅผ ํ†ตํ•ด input data์— Noise๋ผ๊ณ  ์ƒ๊ฐ๋˜๋Š” ๋ถ€๋ถ„์€ ์ œ๊ฑฐํ•˜๊ณ  ์‹ถ๋‹ค๋Š” ๋œป์ด๋‹ค. ์ •๋ฆฌํ•ด์„œ..

๐Ÿ“š Study/AI 2024.07.09

[Algorithm] ๊ทธ๋ฆฌ๋””(Greedy) ์•Œ๊ณ ๋ฆฌ์ฆ˜

# 1. ๋‹น์žฅ ์ข‹์€ ๊ฒƒ๋งŒ ์„ ํƒํ•˜๋Š” ๊ทธ๋ฆฌ๋””๊ทธ๋ฆฌ๋””(Greedy) ์•Œ๊ณ ๋ฆฌ์ฆ˜: ํƒ์š•์  ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ๋œป์€, 'ํ˜„์žฌ ์ƒํ™ฉ์—์„œ ์ง€๊ธˆ ๋‹น์žฅ ์ข‹์€ ๊ฒƒ๋งŒ ๊ณ ๋ฅด๋Š” ๋ฐฉ๋ฒ•'์„ ์˜๋ฏธํ•œ๋‹ค.๋”ฐ๋ผ์„œ, ๋งค์ˆœ๊ฐ„ ๊ฐ€์žฅ ์ข‹์•„๋ณด์ด๋Š” ๊ฒƒ์„ ์„ ํƒํ•˜๊ฒŒ ๋˜๋ฉฐ, ํ˜„์žฌ์˜ ์„ ํƒ์ด ๋ฏธ๋ž˜์— ์–ด๋–ค ์˜ํ–ฅ์„ ๋ฏธ์น ์ง€ ๊ณ ๋ คํ•˜์ง€ ์•Š๋Š” ๊ฒƒ์ด๋‹ค.๋”ฐ๋ผ์„œ ๊ทธ๋ฆฌ๋”” ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ '์ •๋‹น์„ฑ ๋ถ„์„'์ด ์ค‘์š”ํ•˜๋‹ค.๋‹จ์ˆœํžˆ ํ˜„์žฌ best๋ฅผ ์„ ํƒํ•ด๋„ ๊ทธ๊ฒŒ ์ตœ์ ์˜ ํ•ด๊ฐ€ ๋˜๋Š”์ง€ ๊ฒ€ํ† ๊ฐ€ ํ•„์š”ํ•˜๋‹ค๋Š” ์†Œ๋ฆฌ์ด๋‹ค! ์ฝ”๋”ฉ ํ…Œ์ŠคํŠธ์—์„œ ๊ทธ๋ฆฌ๋”” ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ๋ฌธ์ œ์œ ํ˜•์€'์‚ฌ์ „์— ์™ธ์šฐ๊ณ  ์žˆ์ง€ ์•Š์•„๋„ ํ’€ ์ˆ˜ ์žˆ์„ ๊ฐ€๋Šฅ์„ฑ์ด ๋†’์€ ๋ฌธ์ œ ์œ ํ˜•'์ด๋‹ค.๋ฐ˜๋ฉด ์ดํ›„์— ๊ณต๋ถ€ํ•  ์ •๋ ฌ, ์ตœ๋‹จ ๊ฒฝ๋กœ ๋“ฑ์˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜ ์œ ํ˜•์€์ด๋ฏธ ๊ทธ ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ์‚ฌ์šฉ๋ฐฉ๋ฒ•์„ ์ •ํ™•ํ•˜๊ฒŒ ์•Œ์•„์•ผ ํ•˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ๋งŽ๋‹ค.๋˜ํ•œ, ๊ทธ๋ฆฌ๋””๋Š” ๊ธฐ์ค€์— ๋”ฐ๋ผ ์ข‹์€ ๊ฒƒ์„ ์„ ํƒํ•˜๋Š” ์•Œ๊ณ ๋ฆฌ์ฆ˜์ด๋ฏ€๋กœ๋ฌธ..

[Paper Review] Mini-Splatting: Representing Scenes with a Constrained Number of Gaussians

2024๋…„ 3์›”์— arxiv์— ์˜ฌ๋ผ์˜จ ๋…ผ๋ฌธ์œผ๋กœ,๊ธฐ์กด์— 3DGS์˜ ๊ฐ€์šฐ์‹œ์•ˆ ์ˆ˜ ํ˜น์€ ์ฐจ์›์„ ์ค„์ด๊ธฐ ์œ„ํ•œ ์‹œ๋„๋“ค(LightGaussian, Compact3DGS ๋“ฑ..)์˜ ํ•œ๊ณ„๋ฅผ ์–ธ๊ธ‰ํ•˜๋ฉฐ์ƒˆ๋กœ์šด ์•„์ด๋””์–ด๋ฅผ ์ฃผ์žฅํ•œ๋‹ค๋Š” ์ ์—์„œ ํฅ๋ฏธ๋กœ์›Œ ์ฝ๊ฒŒ ๋˜์—ˆ๋‹ค.  Mini-Splatting: Representing Scenes with a Constrained Number of GaussiansIn this study, we explore the challenge of efficiently representing scenes with a constrained number of Gaussians. Our analysis shifts from traditional graphics and 2D computer vision to t..