Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expression: speed up Column.VecEvalReal by using MergeNulls #22190

Closed
Tjianke opened this issue Jan 5, 2021 · 0 comments · Fixed by #22191
Closed

expression: speed up Column.VecEvalReal by using MergeNulls #22190

Tjianke opened this issue Jan 5, 2021 · 0 comments · Fixed by #22191
Labels
type/question The issue belongs to a question.

Comments

@Tjianke
Copy link
Contributor

Tjianke commented Jan 5, 2021

General Question

I found a todo item('speed up the way to manipulate null-bitmaps') in expression/column.go, so I was thinking maybe using Column.MergeNulls() would be better than setting null on every single bit in the bitmap.

In original code, we have to set null for every null element in nullbitmap, using SetNull() method.

        n := input.NumRows()
	src := input.Column(col.Index)
	if col.GetType().Tp == mysql.TypeFloat {
		result.ResizeFloat64(n, false)
		f32s := src.Float32s()
		f64s := result.Float64s()
		for i := range f32s {
			// TODO(zhangyuanjia): speed up the way to manipulate null-bitmaps.
			if src.IsNull(i) {
				result.SetNull(i, true)
			} else {
				f64s[i] = float64(f32s[i])
			}
		}
		return nil
	}

Theoretically it would be more efficient if we replace Column.SetNull() with Column.MergeNulls(), which utilizes bit operation underneath.

Proposed method:

        n := input.NumRows()
	src := input.Column(col.Index)
	if col.GetType().Tp == mysql.TypeFloat {
		result.ResizeFloat64(n, false)
		f32s := src.Float32s()
		f64s := result.Float64s()
                result.MergeNulls(src)
		for i := range f32s {
			if src.IsNull(i) {
				continue
			} 
           		f64s[i] = float64(f32s[i])		
		}
		return nil
	}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/question The issue belongs to a question.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant