Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

通过C来操作日志文件,使用MMAP机制写入日志,会发现在日志文件存在大量的���������字符。 #9

Open
JessonYue opened this issue Apr 10, 2018 · 5 comments

Comments

@JessonYue
Copy link

通过C来操作日志文件,使用MMAP机制写入日志,会发现在日志文件存在大量的���������字符。

@JessonYue
Copy link
Author

补充下:运行demo也会出现这种情况

@zhongxiaofeng
Copy link
Contributor

出现�这种字符是编码显示的问题,正常情况下都是空字符,mmap文件没有写到的地方都是显示这个空字符,是用来占位扩展文件大小作用的,在日志文件上传前可以先清除后面的空占位符

@JessonYue
Copy link
Author

嗯嗯,我在Linux上通过more查看的确是看到很多空的占位符,我不理解为啥在源码里需要设置空的占位符,这点能通过源码修改解决吗?另外一点就是如果在服务器上通过shell删除空字符或是在android端操作都感觉不是很好,能否在MMAP回写数据到磁盘的地方进行修改呢?请给予指点。
ErrInfo *LogWriter::writeLog(JNIEnv *env, const char *logMsg, size_t textSize) {
if (logMsg == NULL || textSize <= 0) {
return nullptr;
}

if (recordPtr == NULL) {
    close(fd);
    return new ErrInfo(WRITE_EXIT, "Error writing log");
}

ErrInfo *errInfo = checkMmapFile();
if (errInfo != NULL) {
    unixMunmap(fd, static_cast<void *>(recordPtr), logPageSize);
    close(fd);
    return errInfo;
}

size_t msgIndex = 0;

while (1) {

    for (; msgIndex < textSize && recordIndex < logPageSize; msgIndex++) {
        recordPtr[recordIndex] = logMsg[msgIndex];
        recordIndex++;
    }

    //当开辟的mmap内存被写满时,需要再开辟一页mmap内存
    if (recordIndex >= logPageSize) {

        ErrInfo *errInfo = unixMunmap(fd, recordPtr, (size_t) logPageSize);
        if (errInfo != NULL) {
            close(fd);
            return errInfo;
        }

        recordPtr = NULL;

        if (access(filePath.c_str(), 0) != 0) {
            close(fd);
            return new ErrInfo(ACCESS_EXIT, "Error calling access file");
        }

        //扩展文件大小
        if (ftruncate(fd, fileSize + logPageSize) == -1) {
            close(fd);
            return new ErrInfo(LSEEK_EXIT, "Error calling ftruncate() to stretch file");
        }

        //移动到文件末尾
        if (lseek(fd, fileSize + logPageSize - 1, SEEK_SET) == -1) {    
            close(fd);
            return new ErrInfo(LSEEK_EXIT, "Error calling lseek() to stretch the file");
        }

       //在文件末尾写入一个字符,达到扩展文件大小的目的    //----------是否这里是产生空白符的根源
        if (write(fd, "", 1) == -1) {
            close(fd);
            return new ErrInfo(WRITE_EXIT, "Error writing last byte of the file");
        }

        this->fileStat.st_size = 0;

        if (fstat(fd, &fileStat) == -1) {
            close(fd);
            return new ErrInfo(FSTAT_EXIT, "Error fstat file");
        }

        if (fileStat.st_size - logPageSize != this->fileSize &&
            fileStat.st_size % logPageSize != 0) {
            close(fd);
            return new ErrInfo(WRITE_EXIT, "Error stretch file when writing");
        }

        this->fileSize = fileStat.st_size;

        void *map = mmap(NULL, static_cast<size_t>(logPageSize), PROT_READ | PROT_WRITE,
                         MAP_SHARED, fd,
                         fileSize - logPageSize);

        if (map == MAP_FAILED || map == NULL) {
            close(fd);
            return new ErrInfo(MMAP_EXIT, "Error mmaping the file");
        }

        recordPtr = static_cast<char *> (map);

        if (recordPtr == NULL) {
            close(fd);
            return new ErrInfo(MMAP_EXIT, "Error cast char*");
        }

        memset(recordPtr, 0, static_cast<size_t >(logPageSize));

        recordIndex = 0;
    } else {
        break;
    }
}

return nullptr;

}

@zhongxiaofeng
Copy link
Contributor

空字符占位,是为了mmap内存映射作用的,没法修改源码,倒是可以在关闭文件前去除这些占位符

@wangyongf
Copy link

空字符的问题是没法避免的,要么就手动清掉,因为 mmap 在申请的时候是一段一段的,每次申请一段内存映射 buffer,系统都会把这部分初始化为全零,然后写入的时候是从头开始填充 buffer,然鹅,在最后上报日志的时候,最后一次申请的 buffer 一定没用完,所以才有了一堆全零。。。。。。。我之前做 mmap 的时候也遇到了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants