Desbloqueando o "Algoritmo" do Twitter - Introdução a Grafos

Fabio Akita April 11, 2023
Video Thumbnail
Fabio Akita Logo

Fabio Akita

View Channel

About

Este é o canal "AKITANDO" onde eu, Fabio Akita, também conhecido como @AkitaOnRails vou falar dos assuntos que sempre gostei. Quem me acompanha no meu blog, e redes sociais como Facebook, Twitter e Instagram já tem uma idéia do que esperar daqui. Eu curto assuntos geek, otaku, novas tecnologias e quero trazer um pouco disso para cá. Obs: este canal NÃO aceita patrocínios. Como o YouTube forçou monetização contra minha vontade, tudo que é arrecadado eu faço doações para projetos educacionais sem fins lucrativos. #anti-woke #anti-SJW

Video Description

O Twitter liberou parte do código que chamou de "The Algorithm", que forma o sistema que popula a aba de recomendações "Para Você". Vamos entender qual é o contexto de ciência da computação e história necessários pra começar a entender esse código todo. E uma pequena introdução à ciência por trás de recomendações e grafos em geral. Finalmente, vou poder mostrar na prática a diferença de "código de curso" e "código de verdade" e exatamente porque software nunca acaba. ## ERRATA Vira e mexe eu falo nó e vértice mas o correto são nó e arestas. Eu estou acostumado a falar em inglês só nodes e edges e traduzi errado em português. ## Capítulos 00:00:00 - Intro 00:00:57 - CAP 1 - Avisos e Descrevendo o Repositório 00:02:51 - CAP 2 - Interpretando Errado: Não Acredite em Blog Posts 00:07:14 - CAP 3 - Derrubando o Meme: Pessoas Idiotas 00:11:30 - CAP 4 - Objetivos do Video: derrubar os Clones. Código de Verdade 00:16:37 - CAP 5 - Entendendo Grafos. Introdução a Relevância 00:21:47 - CAP 6 - Primeira Geração de Relevância: WTF 00:27:49 - CAP 7 - Relevância nos Primórdios da Web: PageRank e HITS 00:35:43 - CAP 8 - Segunda Geração de Relevância: RealGraph e SALSA 00:39:22 - CAP 9 - Escalando com Hadoop e MapReduce: HDFS e Pig 00:43:15 - CAP 10 - Algoritmos Escaláveis de Grafos: Relembrando Mergesort 00:46:58 - CAP 11 - Terceira Geração de Relevância: GraphJet 00:50:36 - CAP 12 - Earlybird: Light Ranker. Apache Lucene 00:51:37 - CAP 13 - Ids para Sistemas Distribuídos: Snowflake 00:55:23 - CAP 14 - Serialização Eficiente sem JSON: Thrift e Protobufs 01:01:07 - CAP 15 - A Controvérsia do "Rails Não Escala". Por que Scala? 01:07:37 - CAP 16 - Introdução a Similarity Clusters: Similaridade de Cosseno 01:16:08 - CAP 17 - Resumindo os demais Projetos: Aplicando o que Aprendemos 01:20:22 - CAP 18 - Qualidade do Código: Mini-Assessment 01:22:43 - CAP 19 - Sendo Educado na Web: Memes Idiotas 01:24:32 - CAP 20 - Refletindo sobre o Código: Lições a Aprender 01:27:31 - Bloopers ## Links * Fio no Twitter (https://twitter.com/AkitaOnRails/status/1641887891327537158?s=20) * Twitter: The Algorithm (https://github.com/twitter/the-algorithm) * Twitter: The Algorithm ML (https://github.com/twitter/the-algorithm-ml) * Twitter's Recommendation Algorithm (https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm) * Steven Tey: # How the Twitter Algorithm works in 2023 (https://steventey.com/blog/twitter-algorithm) * Aakash Gupta: The Real Twitter Files: The Algorithm (https://aakashgupta.substack.com/p/the-real-twitter-files-the-algorithm) * Earlybird: Real-Time Search at Twitter (https://stephenholiday.com/notes/earlybird/) * Twitter Search is Now 3x Faster (https://blog.twitter.com/engineering/en_us/a/2011/twitter-search-is-now-3x-faster) * Paper: GraphJet: Real-Time Content Recommendations at Twitter (https://www.vldb.org/pvldb/vol9/p1281-sharma.pdf) * Paper: RealGraph: User Interaction Prediction at Twitter (https://www.ueo-workshop.com/wp-content/uploads/2014/04/sig-alternate.pdf) * Paper: Real-Time Twitter Recommendation: Online Motif Detection in Large Dynamic Graphs (http://www.vldb.org/pvldb/vol7/p1379-lin.pdf) * Paper: Discovering Similar Users on Twitter (http://snap.stanford.edu/mlg2013/submissions/mlg2013_submission_20.pdf) * GitHub: FlockDB (https://github.com/twitter-archive/flockdb) * GitHub: Cassovary (https://github.com/twitter/cassovary) * GitHub: GraphJet (https://github.com/twitter/GraphJet) * Twitter: Finagle (https://twitter.github.io/finagle/) * Apache Avro (https://avro.apache.org/) * AkitaOnRails: Chatting with Blaine Cook (Twitter) (https://www.akitaonrails.com/2008/6/17/chatting-with-blaine-cook-twitter) * Fireship: Vector databases are so hot right now. WTF are they? (https://www.youtube.com/watch?v=klTvEwg3oJ4) * Slideshare: Restrição == Inovação (https://www.slideshare.net/akitaonrails/devconf-2019-so-carlos) * Elastic: Vector Search (https://www.elastic.co/what-is/vector-search) * Transcript: https://www.akitaonrails.com/2023/04/11/akitando-140-desbloqueando-o-algoritmo-do-twitter-introducao-a-grafos * Podcast: https://podcasters.spotify.com/pod/show/akitando/episodes/Akitando-140---Desbloqueando-o-Algoritmo-do-Twitter---Introduo-a-Grafos-e225io4

You May Also Like

Upgrade Your Everyday

AI-recommended products based on this video

Loading...
Kasa Smart Outdoor Smart Plug by TP-Link (KP400) - Smart WiFi Outlet with 2 Sockets, IP64 Waterproof, Works with Alexa and Google Home, 2.4GHz WiFi Required, No Hub Required, Sunset & Sunrise Offset

Kasa Smart Outdoor Smart Plug by TP-Link (KP400) - Smart WiFi Outlet with 2 Sockets, IP64 Waterproof, Works with Alexa and Google Home, 2.4GHz WiFi Required, No Hub Required, Sunset & Sunrise Offset

(28,846)
$21.99$17.99
FREE delivery Wed, Aug 20 on your first order
1K+ bought in past month
Loading...
Wireless Earbuds, Sports Bluetooth Headphones, 80Hrs Playtime Ear Buds with LED Power Display, Noise Canceling Headset, IPX7 Waterproof Earphones for Workout/Running(Black)

Wireless Earbuds, Sports Bluetooth Headphones, 80Hrs Playtime Ear Buds with LED Power Display, Noise Canceling Headset, IPX7 Waterproof Earphones for Workout/Running(Black)

(299)
$32.99
Prime
Loading...
Wireless Earbuds, Bluetooth 5.4 Headphones HiFi Stereo with 4 HD Mic, 50H Playtime Bluetooth Earbuds with LED Display, IPX7 Waterproof Wireless Earphones, Touch Control Ear Buds,USB-C White

Wireless Earbuds, Bluetooth 5.4 Headphones HiFi Stereo with 4 HD Mic, 50H Playtime Bluetooth Earbuds with LED Display, IPX7 Waterproof Wireless Earphones, Touch Control Ear Buds,USB-C White

(228)
$26.99
Prime
Loading...
Wireless Earbuds, Bluetooth Ear Buds 5.4 Mini HiFi Stereo with 4 ENC Noise Cancelling Mics Wireless Headphones, in Ear Earphones 40H IP7 Waterproof, USB C, Bluetooth Earbuds for Sports, Gym, Rose

Wireless Earbuds, Bluetooth Ear Buds 5.4 Mini HiFi Stereo with 4 ENC Noise Cancelling Mics Wireless Headphones, in Ear Earphones 40H IP7 Waterproof, USB C, Bluetooth Earbuds for Sports, Gym, Rose

(323)
$31.99
FREE delivery Wed, Dec 3 on your first order
500+ bought in past month
Loading...
soundcore P31i by Anker, Real-Time Adaptive Noise Cancelling, Hi-Res Sound, Translation Earbuds, 50H Playtime, Wireless Earbuds, Bluetooth Earphones, Spatial Audio, Fast Charging, IP55 ClimatePartner certified

soundcore P31i by Anker, Real-Time Adaptive Noise Cancelling, Hi-Res Sound, Translation Earbuds, 50H Playtime, Wireless Earbuds, Bluetooth Earphones, Spatial Audio, Fast Charging, IP55 ClimatePartner certified

(105)
$39.99
FREE delivery Thu, Nov 27
Loading...
Brita Stainless Steel Premium Filtering Water Bottle, BPA-Free, Reusable, Insulated, Replaces 300 Plastic Water Bottles, Filter Lasts 2 Months or 40 Gallons, Includes 1 Filter, Carbon - 20 oz.

Brita Stainless Steel Premium Filtering Water Bottle, BPA-Free, Reusable, Insulated, Replaces 300 Plastic Water Bottles, Filter Lasts 2 Months or 40 Gallons, Includes 1 Filter, Carbon - 20 oz.

(2,652)
$81.37$76.36
FREE delivery Sep 29 - Oct 2
Loading...
Simple Modern Filtered Water Bottle | Insulated Stainless-Steel Carbon Filter Travel Water Bottles | Reusable for Clean Drinking Water On The Go | 24oz, Sea Glass Sage

Simple Modern Filtered Water Bottle | Insulated Stainless-Steel Carbon Filter Travel Water Bottles | Reusable for Clean Drinking Water On The Go | 24oz, Sea Glass Sage

(26)
$30.13
FREE delivery Tue, Sep 23 on your first order
Loading...
FITVII Health & Fitness Tracker (Answer/Make Calls), Smart Watch with 24/7 Heart Rate and Blood Pressure, Sleep Tracking Monitor, 120+ Sport Mode Activity Tracker

FITVII Health & Fitness Tracker (Answer/Make Calls), Smart Watch with 24/7 Heart Rate and Blood Pressure, Sleep Tracking Monitor, 120+ Sport Mode Activity Tracker

(198)
$59.99
FREE delivery Sun, Aug 10
100+ bought in past month
Loading...
AYATAHA AYATAHA Smart Watch for Kids, Smartwatch Fitness Tracker for Boys Girls, Children's Activity Watch 37 Sports Modes SMS Notification, HD Full Touchscreen IP67 Waterproof, Blue

AYATAHA AYATAHA Smart Watch for Kids, Smartwatch Fitness Tracker for Boys Girls, Children's Activity Watch 37 Sports Modes SMS Notification, HD Full Touchscreen IP67 Waterproof, Blue

(55)
$41.99
PrimeFREE delivery Sat, Jun 14
Loading...
Iaret Iaret Smart Watch for Women, 1.83" HD Fitness Tracker with 4 Bands, Answer/Make Calls, Heart Rate/Sleep/SpO2/Step Tracking, 100+ Sport Modes, Android/iPhone Compatible Gift (Rose Gold)

Iaret Iaret Smart Watch for Women, 1.83" HD Fitness Tracker with 4 Bands, Answer/Make Calls, Heart Rate/Sleep/SpO2/Step Tracking, 100+ Sport Modes, Android/iPhone Compatible Gift (Rose Gold)

(42)
$59.99
FREE delivery Thu, Jun 19
100+ bought in past month
Loading...
Smart Watch for Men Women 1.8" Fitness Tracker, Bluetooth Call, DIY Dial, Heart Rate Sleep Blood Oxygen Monitor, 100+ Sports Modes, IP68 Waterproof Smartwatch for Android iPhone, Alexa Built-in

Smart Watch for Men Women 1.8" Fitness Tracker, Bluetooth Call, DIY Dial, Heart Rate Sleep Blood Oxygen Monitor, 100+ Sports Modes, IP68 Waterproof Smartwatch for Android iPhone, Alexa Built-in

(980)
$39.99
FREE delivery Sun, Sep 21
300+ bought in past month
Loading...
Hand Warmers 2 Pack, 14000mAh Rechargeable Hand Warmers, Electric Hand Warmer Reusable, Portable Power Bank USB Hand Warmers 4 Levels 8 Heating, Gifts for Raynauds Ski Golf Camping

Hand Warmers 2 Pack, 14000mAh Rechargeable Hand Warmers, Electric Hand Warmer Reusable, Portable Power Bank USB Hand Warmers 4 Levels 8 Heating, Gifts for Raynauds Ski Golf Camping

(683)
$74.08$59.47
FREE delivery Jun 13 - 18
Loading...
Hand Warmers Rechargeable, 10000mAh Electric Heated Gloves Power Bank Portable Graphene Handwarmers Pouch with 3 Levels & Double-Sided Heating for Hunting Camping Golf Xmas Gifts for Women Men Kids

Hand Warmers Rechargeable, 10000mAh Electric Heated Gloves Power Bank Portable Graphene Handwarmers Pouch with 3 Levels & Double-Sided Heating for Hunting Camping Golf Xmas Gifts for Women Men Kids

(651)
$44.99
PrimeFREE delivery Sun, Jun 15
Loading...
2Pack Rechargeable Hand Warmer, 8000mAh Electric Hand Warmer Power Bank, Portable USB-C Hand Warmer for Pocket, Reusable Hand Warmer Up to 8 hrs Each, Warm Gift for Men Women, for Hunting, Camping
Amazon's Choice

2Pack Rechargeable Hand Warmer, 8000mAh Electric Hand Warmer Power Bank, Portable USB-C Hand Warmer for Pocket, Reusable Hand Warmer Up to 8 hrs Each, Warm Gift for Men Women, for Hunting, Camping

(1,128)
$15.99
FREE delivery Sat, Jun 14 on your first order
Loading...
GTOCE Portable Charger,40000mAh Power Bank with 22.5W Fast Charging LED Digital Display Battery Pack with 6 Outputs 2 Inputs, Type C Powerbank Portable Charger for iPhone 16 pro Samsung AirPods,Black

GTOCE Portable Charger,40000mAh Power Bank with 22.5W Fast Charging LED Digital Display Battery Pack with 6 Outputs 2 Inputs, Type C Powerbank Portable Charger for iPhone 16 pro Samsung AirPods,Black

(334)
$42.99
FREE delivery Thu, Oct 16
1K+ bought in past month
Loading...
YMZ Smart Watch for Men Women, IP68 Waterproof, Bluetooth Calling, Fitness Tracker, Sleep Monitoring, 1.85'' DIY Dial, AI Built-in, 100+ Sports Modes, 7 Days Battery, Compatible with Android & iPhones

YMZ Smart Watch for Men Women, IP68 Waterproof, Bluetooth Calling, Fitness Tracker, Sleep Monitoring, 1.85'' DIY Dial, AI Built-in, 100+ Sports Modes, 7 Days Battery, Compatible with Android & iPhones

(331)
$49.99
FREE delivery Thu, Jun 19
1K+ bought in past month
Loading...
Smart Watch for Men Women 1.8" Fitness Tracker, Bluetooth Call, DIY Dial, Heart Rate Sleep Blood Oxygen Monitor, 100+ Sports Modes, IP68 Waterproof Smartwatch for Android iPhone, Alexa Built-in

Smart Watch for Men Women 1.8" Fitness Tracker, Bluetooth Call, DIY Dial, Heart Rate Sleep Blood Oxygen Monitor, 100+ Sports Modes, IP68 Waterproof Smartwatch for Android iPhone, Alexa Built-in

(980)
$39.99
FREE delivery Sun, Sep 21
300+ bought in past month
Loading...
Smart Watch for Men Women with Bluetooth Call, Alexa Built-in, Heart Rate & Sleep Monitor, 1.83" DIY Dial Fitness Watch with 100+ Sport Modes, IP68 Waterproof, Compatible with Android iOS

Smart Watch for Men Women with Bluetooth Call, Alexa Built-in, Heart Rate & Sleep Monitor, 1.83" DIY Dial Fitness Watch with 100+ Sport Modes, IP68 Waterproof, Compatible with Android iOS

(182)
$49.99
FREE delivery Fri, Oct 31
2K+ bought in past month
Loading...
Smart Watch for Men Women with Bluetooth Call, Alexa Built-in, Heart Rate & Sleep Monitor, 1.83" DIY Dial Fitness Watch with 100+ Sport Modes, IP68 Waterproof, Compatible with Android iOS
Best Seller

Smart Watch for Men Women with Bluetooth Call, Alexa Built-in, Heart Rate & Sleep Monitor, 1.83" DIY Dial Fitness Watch with 100+ Sport Modes, IP68 Waterproof, Compatible with Android iOS

(142)
$44.99
FREE delivery Tue, Oct 21
Loading...
Hydroponics Growing System Indoor Garden - Herb Garden with Grow Light, 15 Pods Stainless Steel Indoor Garden Kit, Auto Timer, Gardening Gift for All Ages

Hydroponics Growing System Indoor Garden - Herb Garden with Grow Light, 15 Pods Stainless Steel Indoor Garden Kit, Auto Timer, Gardening Gift for All Ages

(24)
$206.57$200.61
FREE delivery Jun 13 - 17
Loading...
Umbra Triflora Hanging Planter for Window, Indoor Herb Garden, Set of 5, White/Black

Umbra Triflora Hanging Planter for Window, Indoor Herb Garden, Set of 5, White/Black

(7,808)
$71.52
PrimeFREE delivery Sat, Jun 14
Loading...
Large Hydroponics Growing System 14 Pods, Indoor Herb Garden with LED Grow Light, 5L Water Tank, Hydroponic Grow Kit with 3 Auto-Timers, Rotatable Light Panel and Child Lock for Home School Gardening

Large Hydroponics Growing System 14 Pods, Indoor Herb Garden with LED Grow Light, 5L Water Tank, Hydroponic Grow Kit with 3 Auto-Timers, Rotatable Light Panel and Child Lock for Home School Gardening

(119)
$167.81
FREE delivery Jun 16 - 20
Loading...
Hanging Planter Hanging Plant Holder, 6 Inch 4 Indoor Plant Pots, Wall/Window Plant Hanger Indoor Herb Garden

Hanging Planter Hanging Plant Holder, 6 Inch 4 Indoor Plant Pots, Wall/Window Plant Hanger Indoor Herb Garden

(269)
$35.99$28.90
PrimeFREE delivery Sat, Jun 14
Loading...
Ninja Air Fryer Pro 5-in-1, 5 QT Capacity, Air Fry, Bake, Roast, Reheat, and Dehydrate, Air Crisp Technology, Nonstick Basket & Crisper Plate, Grey, AF142C

Ninja Air Fryer Pro 5-in-1, 5 QT Capacity, Air Fry, Bake, Roast, Reheat, and Dehydrate, Air Crisp Technology, Nonstick Basket & Crisper Plate, Grey, AF142C

(650)
$100.99
FREE delivery Thu, Oct 16
500+ bought in past month
Loading...
Glass Oil Sprayer for Cooking, 180ml Olive Oil Sprayer Bottle with Brush, Olive Oil Spray Mister, Kitchen Gadgets Accessories for Air Fryer, Canola Oil Spritzer (GREY)

Glass Oil Sprayer for Cooking, 180ml Olive Oil Sprayer Bottle with Brush, Olive Oil Spray Mister, Kitchen Gadgets Accessories for Air Fryer, Canola Oil Spritzer (GREY)

(2,436)
$7.99
FREE delivery Mon, Jun 16 on your first order
Loading...
Hi.FANCY Portable Laptop Stand with Dual Cooling Fans for 14-17inch Laptops, Grey, 23.5 x 25.9 x 0.95cm

Hi.FANCY Portable Laptop Stand with Dual Cooling Fans for 14-17inch Laptops, Grey, 23.5 x 25.9 x 0.95cm

(0)
$36.99
FREE delivery Aug 18 - 28
Loading...
Laptop Stand for Desk, Adjustable Laptop Riser ABS+Silicone Foldable Portable Laptop Holder, Ventilated Cooling Notebook Stand for 10-15.6” Laptops,Tablet-Black

Laptop Stand for Desk, Adjustable Laptop Riser ABS+Silicone Foldable Portable Laptop Holder, Ventilated Cooling Notebook Stand for 10-15.6” Laptops,Tablet-Black

(11,679)
$14.99
FREE delivery Sat, Jun 14 on your first order
500+ bought in past month
Loading...
JETech 5 in 1 Case for Samsung Galaxy S25 Ultra 5G with 2-Pack Each Tempered Glass Screen Protector and Camera Lens Protector, Non-Yellowing Shockproof Bumper Phone Cover (Clear)

JETech 5 in 1 Case for Samsung Galaxy S25 Ultra 5G with 2-Pack Each Tempered Glass Screen Protector and Camera Lens Protector, Non-Yellowing Shockproof Bumper Phone Cover (Clear)

(197)
$15.99
FREE delivery Sat, Jun 14 on your first order
100+ bought in past month
Loading...
TAURI for iPhone 17 Pro Max Case 6.9" with 1-Pack Screen Protector, Camera Lens Full Protection, Military-Grade Protection, Shockproof Transparent Back Bumper Phone Cover - Clear Global Recycled Standard

TAURI for iPhone 17 Pro Max Case 6.9" with 1-Pack Screen Protector, Camera Lens Full Protection, Military-Grade Protection, Shockproof Transparent Back Bumper Phone Cover - Clear Global Recycled Standard

(0)
$10.76
FREE delivery Thu, Oct 16 on your first order
2K+ bought in past month
Loading...
TAURI for iPhone 17 Pro Case 6.3" with 1-Pack Screen Protector, Camera Lens Full Protection, Military-Grade Protection, Shockproof Transparent Back Bumper Phone Cover - Clear Global Recycled Standard
Best Seller

TAURI for iPhone 17 Pro Case 6.3" with 1-Pack Screen Protector, Camera Lens Full Protection, Military-Grade Protection, Shockproof Transparent Back Bumper Phone Cover - Clear Global Recycled Standard

(61,696)
$10.76
Prime
2K+ bought in past month
Loading...
JOINPAYA 1Set Rechargeable Hand Warmer Hand Heater for Winter Heating Levels Compact

JOINPAYA 1Set Rechargeable Hand Warmer Hand Heater for Winter Heating Levels Compact

(0)
$9.29
FREE delivery Jun 20 - Jul 4
Loading...
Shakven Rechargeable Hand Warmer | Cute Comfortable Portable Hand Warmers,Ergonomic Adjustable Energy-Efficient Small Heater for Travel, Outdoor, Winter

Shakven Rechargeable Hand Warmer | Cute Comfortable Portable Hand Warmers,Ergonomic Adjustable Energy-Efficient Small Heater for Travel, Outdoor, Winter

(0)
$26.93
FREE delivery Jun 19 - Jul 2
Loading...
OCOOPA IP45 Waterproof Hand Warmer Rechargeable, Up to 15hrs Heat,10000mAh Durable Quick Charge Electric Hand Heater, PD Compatible, 3 Levels for Outdoors, Heavy Duty, H01-PD PRO

OCOOPA IP45 Waterproof Hand Warmer Rechargeable, Up to 15hrs Heat,10000mAh Durable Quick Charge Electric Hand Heater, PD Compatible, 3 Levels for Outdoors, Heavy Duty, H01-PD PRO

(50)
$39.99
PrimeFREE delivery Sat, Jun 14
Loading...
OCOOPA IP45 Waterproof Hand Warmer Rechargeable, Up to 15hrs Heat,10000mAh Durable Quick Charge Electric Hand Heater, PD Compatible, 3 Levels for Outdoors, Heavy Duty, H01-PD PRO

OCOOPA IP45 Waterproof Hand Warmer Rechargeable, Up to 15hrs Heat,10000mAh Durable Quick Charge Electric Hand Heater, PD Compatible, 3 Levels for Outdoors, Heavy Duty, H01-PD PRO

(649)
$39.99
PrimeFREE delivery Sat, Jun 14