[Saw a PATH_MAX which of course would need to go. 24 Nov 1998: John Cavanaugh writes: You probably want to remove it, since I just realized I mailed a diff against the wrong working file. The patch as it stands wont work. I skipped resending the correct one since I am going to do more invasive work on history.c this weekend -kingdon] Date: Tue, 24 Nov 1998 17:14:07 -0800 From: John Cavanaugh To: bug-cvs@gnu.org Subject: Questions/Comments on another history.c patch Content-Type: multipart/mixed; boundary=BOKacYhQ+x31HxR3 --BOKacYhQ+x31HxR3 Content-Type: text/plain; charset=us-ascii Today while diagnosing a cvs core dump during history queries I found (much to my dismay) that when processing the *entire* history file is sucked into memory. This is a big problem if you have large repositories with lots of users (which we have) since cvs goes off trying to malloc 70+Meg of memory. Attached is a patch that processes the history file line by line. Its actually simpler than reading the whole thing into memory... Ive tested it on hpux & linux with no problems (Ill test on WinNT tomorrow). The code I modified is ancient (circa 1994) so Im not sure if I am voiding some assumptions. Maybe someone else can help clarify. Specifically, is there any reason why the history file should be read as binary?? Was there a specific reason why the code was written to read the whole thing into memory? PS. This patch isnt really complete, I sort of just punted on making the mods in fill_hrec (it really should be changed to a void routine since it doesnt need to return anything anymore) ----------------------------------------------------------------------- John Cavanaugh Hewlett-Packard Company Project Engineer 1400 Fountaingrove Pkwy EESof Division Santa Rosa, CA 95403-1799 Email: cavanaug@sr.hp.com Phone: 707-577-4780 707-577-3948 (Fax) ----------------------------------------------------------------------- When schemes are laid in advance, it is surprising how often the circumstances fit in with them. -- William Osler ----------------------------------------------------------------------- --BOKacYhQ+x31HxR3 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="history.c_patch" Index: history.c =================================================================== RCS file: /cvs/tools/src/cvs-src/src/history.c,v retrieving revision 1.2 diff -c -r1.2 history.c *** history.c 1998/11/24 07:33:43 1.2 --- history.c 1998/11/25 01:03:26 *************** *** 998,1006 **** memset ((char *) hr, 0, sizeof (*hr)); while (isspace (*line)) line++; - if (!(rtn = strchr (line, '\n'))) - return (""); - *rtn++ = '\0'; hr->type = line++; (void) sscanf (line, "%lx", &date); --- 998,1003 ---- *************** *** 1035,1042 **** * (history record) array elements with the ones we need to print. * * Logic: ! * - Read the whole history file into a single buffer. ! * - Walk through the buffer, parsing lines out of the buffer. * 1. Split line into pointer and integer fields in the "next" hrec. * 2. Apply tests to the hrec to see if it is wanted. * 3. If it *is* wanted, bump the hrec pointer down by one. --- 1032,1038 ---- * (history record) array elements with the ones we need to print. * * Logic: ! * - Walk through the history file line by line * 1. Split line into pointer and integer fields in the "next" hrec. * 2. Apply tests to the hrec to see if it is wanted. * 3. If it *is* wanted, bump the hrec pointer down by one. *************** *** 1045,1087 **** read_hrecs (fname) char *fname; { ! char *cp, *cp2; ! int i, fd; struct hrec *hr; - struct stat st_buf; ! if ((fd = CVS_OPEN (fname, O_RDONLY | OPEN_BINARY)) < 0) error (1, errno, "cannot open history file: %s", fname); - if (fstat (fd, &st_buf) < 0) - error (1, errno, "can't stat history file"); - - /* Exactly enough space for lines data */ - if (!(i = st_buf.st_size)) - error (1, 0, "history file is empty"); - cp = xmalloc (i + 2); - - if (read (fd, cp, i) != i) - error (1, errno, "cannot read log file"); - (void) close (fd); - - if (*(cp + i - 1) != '\n') - { - *(cp + i) = '\n'; /* Make sure last line ends in '\n' */ - i++; - } - *(cp + i) = '\0'; - for (cp2 = cp; cp2 - cp < i; cp2++) - { - if (*cp2 != '\n' && !isprint (*cp2)) - *cp2 = ' '; - } - hrec_max = HREC_INCREMENT; hrec_head = (struct hrec *) xmalloc (hrec_max * sizeof (struct hrec)); ! while (*cp) { if (hrec_count == hrec_max) { struct hrec *old_head = hrec_head; --- 1041,1061 ---- read_hrecs (fname) char *fname; { ! char cp[PATH_MAX]; ! int i; ! FILE *fd; struct hrec *hr; ! if ((fd = CVS_FOPEN (fname, "r")) < 0) error (1, errno, "cannot open history file: %s", fname); hrec_max = HREC_INCREMENT; hrec_head = (struct hrec *) xmalloc (hrec_max * sizeof (struct hrec)); ! while (!feof(fd)) { + fscanf(fd,"%s",cp); + if (hrec_count == hrec_max) { struct hrec *old_head = hrec_head; *************** *** 1099,1105 **** } hr = hrec_head + hrec_count; ! cp = fill_hrec (cp, hr); /* cp == next line or '\0' at end of buffer */ if (select_hrec (hr)) hrec_count++; --- 1073,1079 ---- } hr = hrec_head + hrec_count; ! fill_hrec (cp, hr); if (select_hrec (hr)) hrec_count++; *************** *** 1122,1127 **** --- 1096,1103 ---- hrec_count -= (last_backto - hrec_head); hrec_head = last_backto; } + + fclose(fd); } /* Utility program for determining whether "find" is inside "string" */ --BOKacYhQ+x31HxR3--