@@ -22,7 +22,7 @@ Databricks <br/> Connect
22
22
:::{.columns}
23
23
:::{.column width="50%"}
24
24
25
- :::{.custom-subtitle }
25
+ :::{.custom-smaller }
26
26
:::{.incremental1}
27
27
- Spark Connect, offers ** true** remote connectivity
28
28
- Uses ** gRPC** to as the communication interface
@@ -37,22 +37,10 @@ Databricks <br/> Connect
37
37
38
38
![ ] ( assets/databricks-connect/grpc ) {.absolute top="264" left="900" width="670"}
39
39
40
-
41
40
## [ Databricks Connect] {style="color:#666 ;"} {background-image="assets/background/slide-light.svg" background-size="1700px" background-color="white"}
42
41
43
- :::{.columns}
44
- :::{.column width="37%"}
45
-
46
- :::{.custom-subtitle}
47
- :::{.incremental1}
48
- - ` databricks-connect ` integrates with gRPC, by wrapping ` pyspark `
49
- - ` pyspark ` is the most developed ** Spark Connect** interface
50
- :::
51
- :::
52
-
53
- :::
54
- :::{.column width="60%"}
55
- :::
42
+ :::{.custom-subtitle .custom-smaller}
43
+ ` databricks-connect ` integrates with gRPC via ` pyspark `
56
44
:::
57
45
58
46
![ ] ( assets/databricks-connect/python.png ) {.absolute top="264" left="572" width="998"}
@@ -61,29 +49,23 @@ Databricks <br/> Connect
61
49
62
50
![ ] ( assets/posit-databricks.png ) {.absolute top="-10" left="1430" width="180"}
63
51
64
- :::{.columns}
65
- :::{.column width="4%"}
66
- :::
67
- :::{.column width="96%"}
68
- [ ` sparklyr ` integrates with ` databricks-connect ` via ` reticulate ` ] {style="font-size:54px;line-height:1;font-weight:400;color:#666 ;"}
69
- :::
52
+ :::{.custom-subtitle}
53
+ ` sparklyr ` integrates with ` databricks-connect ` via ` reticulate `
70
54
:::
71
55
72
-
73
-
74
56
![ ] ( assets/databricks-connect/db-connect.png ) {.absolute top="200" left="70" width="1500"}
75
57
76
58
## [ Why not just use 'reticulate'?] {style="color:#666 ;"} {background-image="assets/background/slide-light.svg" background-size="1700px" background-color="white"}
77
59
78
- ![ ] ( assets/posit-databricks.png ) {.absolute top="-10" left="1430" width="180"}
79
-
80
- [ ** sparklyr** extends functionality and user experience:] {style="font-size:65px;line-height:1;font-weight:400;color:#666 ;"}
81
-
60
+ :::{.custom-subtitle}
61
+ ` sparklyr ` extends functionality and user experience
62
+ :::
82
63
83
64
:::{.columns}
84
- :::{.column width="45%"}
85
-
86
- :::{.custom-subtitle}
65
+ :::{.column width="20%"}
66
+ :::
67
+ :::{.column width="70%"}
68
+ :::{.custom-smaller}
87
69
:::{.incremental1}
88
70
- ` dplyr ` back-end
89
71
- ` DBI ` back-end
@@ -92,34 +74,19 @@ Databricks <br/> Connect
92
74
:::
93
75
:::
94
76
95
- :::
96
- :::{.column width="55%"}
97
- :::{.code-slim-35}
98
- ``` r
99
- library(sparklyr )
100
- sc <- spark_connect(method = " databricks_connect" )
101
-
102
- trips <- tbl(sc , I(" samples.nyctaxi.trips" ))
103
-
104
- trips | >
105
- group_by(pickup_zip ) | >
106
- summarise(
107
- count = n(),
108
- avg_distance = mean(trip_distance )
109
- )
110
- ```
111
- :::
112
77
:::
113
78
:::
114
79
80
+ ![ ] ( assets/posit-databricks.png ) {.absolute top="-10" left="1430" width="180"}
81
+
115
82
## [ Getting started] {style="color:#666 ;"} {background-image="assets/background/slide-light.svg" background-size="1700px" background-color="white"}
116
83
117
84
![ ] ( assets/posit-databricks.png ) {.absolute top="-10" left="1430" width="180"}
118
85
119
86
:::{.columns}
120
87
:::{.column width="42%"}
121
88
122
- :::{.custom-subtitle }
89
+ :::{.custom-smaller }
123
90
:::{.incremental1}
124
91
- Python 3.10+
125
92
- A Python environment with ` databricks-connect ` and its dependencies
@@ -129,14 +96,17 @@ trips |>
129
96
130
97
:::
131
98
:::{.column width="58%"}
132
- :::{.code-slim-35}
99
+ :::{.custom-smaller}
100
+ <br />
133
101
``` r
134
102
install.packages(" pysparklyr" )
103
+
135
104
library(sparklyr )
105
+
136
106
sc <- spark_connect(
137
- cluster_id = " 1026-175310-7cpsh3g8 " ,
138
- method = " databricks_connect"
139
- )
107
+ cluster_id = " [cluster's id] " ,
108
+ method = " databricks_connect"
109
+ )
140
110
```
141
111
:::
142
112
:::
0 commit comments